Spark Bench

DeepSeek V4.1 Flash 1017 EXL3-EAQ 2.9 bpw: TensorFold native, two Sparks (TP=2), 262K, thinking off

2× DGX Spark · TensorFold ·

Tested configuration

Hardware
2x NVIDIA DGX Spark (128GB each)
Serving runtime
TensorFold
Runtime version or build
TensorFold (native Zig, TP=2) built locally by the recipe's prepare.sh with PULL=0 (no registry image was pulled)
Weight quantization or precision
EXL3-EAQ 2.9 bpw
Benchmark date
Recipe credit
MiaAI-Lab DeepSeek V4.1 Flash 1017 recipeBy MiaAI-Lab1afc8910b76574aa6b844a08ec9643d14d57089a

Decode sweep

~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.

ConcurrencyOutput tok/sTTFT p95 (s)
165.620.48
290.250.86
4119.531.28
8120.5619.48
12123.0635.61
16123.1852.79

Prefill and context

Configured request window
262,144 tokens
Longest tested prompt
245,760 target input tokens
Validated retrieval
≈240K input tokens · passkey passed

Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.

Workload (c1)Input tok/sTTFT p95 (s)
4,096 input tokens1,218.502.32
32,768 input tokens1,719.7219.56
131,072 input tokens1,095.33127.77
245,760 input tokensPasskey prompt1,959.11125.17

Sustained load

20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.

Aggregate throughput
103.17 output tok/s
TTFT p95
26.40 s
Completed requests
248
Request errors
0

Serving regression checks 13/13 passed

Tool-use evaluation

Incomplete pilot · 98.6% completionSafety gate passed

Comparisons require complete results with 100% completion under the same protocol ID. This run is retained as individual evidence.

Protocol ID
tool-eval-standard-v1-b60e2f0
Workload
69 mock-tool scenarios per trial · 3 trials
Mean score
90.0 / 100
Pass@3 · passed at least once
83.8%
Pass³ · passed every trial
83.8%

Read the tool-eval report and scenario findings

Other configurations for DeepSeek V4.1 Flash 1017 EXL3-EAQ