Spark Bench

Qwen3.8 Flash Next: updated MiaAI recipe with NVIDIA NVFP4 and FP8 KV

2× DGX Spark · vLLM ·

Tested configuration

Hardware
2x NVIDIA DGX Spark (128GB each)
Serving runtime
vLLM
Runtime version or build
vLLM 0.1.dev20073+g8e685d198, image vllm/vllm-openai:qwen38-flash-next, digest sha256:fc120ece0a388cc0aa1caad4a9f1cd92113484ab7ec2fd0efadd62585be05bf8.
Weight quantization or precision
NVFP4
Benchmark date
Recipe credit
MiaAI-Lab updated dual-Spark recipeBy MiaAI-Labc2325b22602b51a5faf55fc2bebccc34f3f80b9f

Decode sweep

~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.

ConcurrencyOutput tok/sTTFT p95 (s)
147.340.40
282.531.05
4133.491.43
8198.091.54
12203.5219.81
16207.7721.27

Prefill and context

Configured request window
262,144 tokens
Longest tested prompt
245,760 target input tokens
Validated retrieval
≈240K input tokens · passkey passed

Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.

Workload (c1)Input tok/sTTFT p95 (s)
4,096 input tokens1,277.501.62
32,768 input tokens2,615.3211.14
131,072 input tokens2,669.1847.69
245,760 input tokensPasskey prompt2,445.5699.22

Sustained load

20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.

Aggregate throughput
197.42 output tok/s
TTFT p95
1.11 s
Completed requests
468
Request errors
0

Serving regression checks 13/13 passed

Tool-use evaluation

Unavailable · no published tool-eval result for this configuration.

Other configurations for Qwen3.8-Flash-Next NVFP4