DeepSeek V4 Flash Vision-Exp on the Anemll DSpark vLLM runtime, two Sparks (TP=2), thinking off
2× DGX Spark · vLLM (Anemll DSpark 0.1.1) ·
Tested configuration
- Hardware
- 2x NVIDIA DGX Spark (128GB each)
- Serving runtime
- vLLM (Anemll DSpark 0.1.1)
- Runtime version or build
- the recipe's Anemll dspark-vllm-gx10 0.1.1 runtime, built locally from Anemll/dspark-vllm-gx10 v0.1.1 (47503f8e38da): upstream vLLM 752a3a504485 (docker/Dockerfile --target vllm-openai, torch_cuda_arch_list=12.1a), then the Anemll runtime layer (FlashInfer 0472b9b3f2fb, b12x 7dc6fb8fcc64, CUTLASS DSL 4.5.2). The prebuilt ghcr.io/anemll/dspark-vllm-gx10:0.1.1 image was not used.
- Weight quantization or precision
- Unavailable
- Benchmark date
- Recipe credit
- MiaAI-Lab DeepSeek-v4-Flash DSpark 2x recipeBy MiaAI-Lab
a8b4636710068fd30edb98e1365d567086cc208e
Decode sweep
~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.
| Concurrency | Output tok/s | TTFT p95 (s) |
|---|---|---|
| 1 | 25.93 | 0.69 |
| 2 | 47.04 | 0.76 |
| 4 | 65.24 | 1.01 |
| 8 | 79.91 | 26.29 |
| 12 | 74.83 | 45.51 |
| 16 | 78.15 | 81.35 |
Prefill and context
- Configured request window
- 1,048,576 tokens
- Longest tested prompt
- 245,760 target input tokens
- Validated retrieval
- ≈240K input tokens · passkey passed
Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.
| Workload (c1) | Input tok/s | TTFT p95 (s) |
|---|---|---|
| 4,096 input tokens | 722.55 | 3.78 |
| 32,768 input tokens | 1,334.30 | 23.04 |
| 131,072 input tokens | 1,150.22 | 111.94 |
| 245,760 input tokensPasskey prompt | 1,325.83 | 185.15 |
Sustained load
20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.
- Aggregate throughput
- 84.53 output tok/s
- TTFT p95
- 23.27 s
- Completed requests
- 205
- Request errors
- 0
Serving regression checks 13/13 passed
Tool-use evaluation
Complete · 100% completionSafety gate failed
Compare only against complete results with 100% completion under the same protocol ID.
- Protocol ID
tool-eval-standard-v1-b60e2f0- Workload
- 69 mock-tool scenarios per trial · 3 trials
- Mean score
- 91.7 / 100
- Pass@3 · passed at least once
- 94.2%
- Pass³ · passed every trial
- 84.1%