DeepSeek V4 Flash 0731 SparkInfer single-Spark serving baseline
1× DGX Spark · vLLM + SparkInfer ·
Tested configuration
- Hardware
- 1x NVIDIA DGX Spark (128GB)
- Serving runtime
- vLLM + SparkInfer
- Runtime version or build
- vLLM with SparkInfer/B12X container ghcr.io/0xsero/deepseek-v4-flash-0731-spark-sparkinfer@sha256:2e077489a83a0360952828051fe7f7a32c1801e5ce8436d85f7267583d614ff4
- Weight quantization or precision
- EXL3
- Benchmark date
- Recipe credit
- MiaAI-Lab DeepSeek V4 Flash single-Spark recipeBy MiaAI-Lab
d4ba142bc1d971eb73a911e207e3e963bbb3c455
Decode sweep
~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.
| Concurrency | Output tok/s | TTFT p95 (s) |
|---|---|---|
| 1 | 29.62 | 0.54 |
| 2 | 31.03 | 21.04 |
| 4 | 31.44 | 54.29 |
| 8 | 31.08 | 118.29 |
| 12 | 29.40 | 195.33 |
| 16 | 29.60 | 263.50 |
Prefill and context
- Configured request window
- 384,000 tokens
- Longest tested prompt
- 245,760 target input tokens
- Validated retrieval
- ≈240K input tokens · passkey passed
Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.
| Workload (c1) | Input tok/s | TTFT p95 (s) |
|---|---|---|
| 4,096 input tokens | 653.49 | 3.95 |
| 32,768 input tokens | 1,046.89 | 28.94 |
| 131,072 input tokens | 1,020.75 | 126.73 |
| 245,760 input tokensPasskey prompt | 947.57 | 259.00 |
Sustained load
20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.
- Aggregate throughput
- 26.89 output tok/s
- TTFT p95
- 150.18 s
- Completed requests
- 71
- Request errors
- 0
Serving regression checks 13/13 passed
Tool-use evaluation
Unavailable · no published tool-eval result for this configuration.