Spark Bench

DeepSeek V4.1 Flash EXL3 2.9 bpw two-Spark serving baseline

2× DGX Spark · vLLM ·

Tested configuration

Hardware
2x NVIDIA DGX Spark (128GB each)
Serving runtime
vLLM
Runtime version or build
vLLM 0.1.dev20904+g179dd0fa9 · sha256:2f0cf3adc0f989c1d446be274df864eb799630175f604c3b22b71b7205971dce
Weight quantization or precision
EXL3 2.9 bpw
Benchmark date
Recipe credit
MiaAI-Lab DeepSeek V4.1 EXL3 recipeBy MiaAI-Lab979e68a62c90b24d928f5638596e0ceed90e9f34

Decode sweep

~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.

ConcurrencyOutput tok/sTTFT p95 (s)
127.420.85
244.490.97
436.7133.09
841.4882.41
1238.30136.85
1636.46208.86

Prefill and context

Configured request window
600,000 tokens
Longest tested prompt
245,760 target input tokens
Validated retrieval
≈240K input tokens · passkey passed

Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.

Workload (c1)Input tok/sTTFT p95 (s)
4,096 input tokens400.598.22
32,768 input tokens559.9557.14
131,072 input tokens574.38228.63
245,760 input tokensPasskey prompt601.70408.37

Sustained load

20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.

Aggregate throughput
35.31 output tok/s
TTFT p95
97.59 s
Completed requests
89
Request errors
0

Serving regression checks 13/13 passed

Tool-use evaluation

Unavailable · no published tool-eval result for this configuration.