Qwen3.8 Flash Next: updated MiaAI recipe with NVIDIA NVFP4 and FP8 KV
2× DGX Spark · vLLM ·
Tested configuration
- Hardware
- 2x NVIDIA DGX Spark (128GB each)
- Serving runtime
- vLLM
- Runtime version or build
- vLLM 0.1.dev20073+g8e685d198, image vllm/vllm-openai:qwen38-flash-next, digest sha256:fc120ece0a388cc0aa1caad4a9f1cd92113484ab7ec2fd0efadd62585be05bf8.
- Weight quantization or precision
- NVFP4
- Benchmark date
- Recipe credit
- MiaAI-Lab updated dual-Spark recipeBy MiaAI-Lab
c2325b22602b51a5faf55fc2bebccc34f3f80b9f
Decode sweep
~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.
| Concurrency | Output tok/s | TTFT p95 (s) |
|---|---|---|
| 1 | 47.34 | 0.40 |
| 2 | 82.53 | 1.05 |
| 4 | 133.49 | 1.43 |
| 8 | 198.09 | 1.54 |
| 12 | 203.52 | 19.81 |
| 16 | 207.77 | 21.27 |
Prefill and context
- Configured request window
- 262,144 tokens
- Longest tested prompt
- 245,760 target input tokens
- Validated retrieval
- ≈240K input tokens · passkey passed
Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.
| Workload (c1) | Input tok/s | TTFT p95 (s) |
|---|---|---|
| 4,096 input tokens | 1,277.50 | 1.62 |
| 32,768 input tokens | 2,615.32 | 11.14 |
| 131,072 input tokens | 2,669.18 | 47.69 |
| 245,760 input tokensPasskey prompt | 2,445.56 | 99.22 |
Sustained load
20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.
- Aggregate throughput
- 197.42 output tok/s
- TTFT p95
- 1.11 s
- Completed requests
- 468
- Request errors
- 0
Serving regression checks 13/13 passed
Tool-use evaluation
Unavailable · no published tool-eval result for this configuration.