Spark Bench

DeepSeek V4.1 Flash 1017 EXL3-EAQ 2.9 bpw: TensorFold native, two Sparks (TP=2), 262K, recipe 27741e17, thinking off

2× DGX Spark · TensorFold ·

Tested configuration

Hardware
2x NVIDIA DGX Spark (128GB each)
Serving runtime
TensorFold
Runtime version or build
TensorFold (native Zig, TP=2) built locally with the recipe's build arguments and PULL=0 (no registry image was pulled)
Weight quantization or precision
EXL3-EAQ 2.9 bpw
Benchmark date
Recipe credit
MiaAI-Lab DeepSeek V4.1 Flash 1017 recipeBy MiaAI-Lab27741e176a9e5b98c10c9c7d012fd3e4fcdc19b6

Decode sweep

~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.

ConcurrencyOutput tok/sTTFT p95 (s)
167.500.51
293.970.64
4124.871.24
8124.1918.91
12124.3335.64
16125.8751.85

Prefill and context

Configured request window
262,144 tokens
Longest tested prompt
245,760 target input tokens
Validated retrieval
≈240K input tokens · passkey passed

Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.

Workload (c1)Input tok/sTTFT p95 (s)
4,096 input tokens1,298.282.20
32,768 input tokens2,008.0715.27
131,072 input tokens2,008.2164.30
245,760 input tokensPasskey prompt2,037.39120.40

Sustained load

20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.

Aggregate throughput
108.73 output tok/s
TTFT p95
22.45 s
Completed requests
261
Request errors
0

Serving regression checks 13/13 passed

Tool-use evaluation

Incomplete pilot · 98.6% completionSafety gate passed

Comparisons require complete results with 100% completion under the same protocol ID. This run is retained as individual evidence.

Protocol ID
tool-eval-standard-v1-b60e2f0
Workload
69 mock-tool scenarios per trial · 3 trials
Mean score
90.0 / 100
Pass@3 · passed at least once
82.4%
Pass³ · passed every trial
82.4%

Read the tool-eval report and scenario findings

Other configurations for DeepSeek V4.1 Flash 1017 EXL3-EAQ