DeepSeek V4.1 Flash EXL3 2.9 bpw two-Spark serving baseline
2× DGX Spark · vLLM ·
Tested configuration
- Hardware
- 2x NVIDIA DGX Spark (128GB each)
- Serving runtime
- vLLM
- Runtime version or build
- vLLM 0.1.dev20904+g179dd0fa9 · sha256:2f0cf3adc0f989c1d446be274df864eb799630175f604c3b22b71b7205971dce
- Weight quantization or precision
- EXL3 2.9 bpw
- Benchmark date
- Recipe credit
- MiaAI-Lab DeepSeek V4.1 EXL3 recipeBy MiaAI-Lab
979e68a62c90b24d928f5638596e0ceed90e9f34
Decode sweep
~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.
| Concurrency | Output tok/s | TTFT p95 (s) |
|---|---|---|
| 1 | 27.42 | 0.85 |
| 2 | 44.49 | 0.97 |
| 4 | 36.71 | 33.09 |
| 8 | 41.48 | 82.41 |
| 12 | 38.30 | 136.85 |
| 16 | 36.46 | 208.86 |
Prefill and context
- Configured request window
- 600,000 tokens
- Longest tested prompt
- 245,760 target input tokens
- Validated retrieval
- ≈240K input tokens · passkey passed
Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.
| Workload (c1) | Input tok/s | TTFT p95 (s) |
|---|---|---|
| 4,096 input tokens | 400.59 | 8.22 |
| 32,768 input tokens | 559.95 | 57.14 |
| 131,072 input tokens | 574.38 | 228.63 |
| 245,760 input tokensPasskey prompt | 601.70 | 408.37 |
Sustained load
20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.
- Aggregate throughput
- 35.31 output tok/s
- TTFT p95
- 97.59 s
- Completed requests
- 89
- Request errors
- 0
Serving regression checks 13/13 passed
Tool-use evaluation
Unavailable · no published tool-eval result for this configuration.