GLM-5.3 Flash NVFP4 vLLM two-Spark low-reasoning baseline
2× DGX Spark · vLLM ·
Tested configuration
- Hardware
- 2x NVIDIA DGX Spark (128GB each)
- Serving runtime
- vLLM
- Runtime version or build
- vLLM in MiaAI-Lab image mia/glm53-flash-spark:mm-ray-v1
- Weight quantization or precision
- NVFP4
- Benchmark date
- Recipe credit
- MiaAI-Lab GLM-5.3 Flash NVFP4 recipeBy MiaAI-Lab
aed98a13ca75140d2691cc5c651ea5817d9a3e44
Decode sweep
~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.
| Concurrency | Output tok/s | TTFT p95 (s) |
|---|---|---|
| 1 | 14.20 | 0.45 |
| 2 | 27.40 | 0.97 |
| 4 | 40.87 | 6.43 |
| 8 | 65.94 | 1.88 |
| 12 | 63.04 | 69.96 |
| 16 | 66.18 | 63.40 |
Prefill and context
- Configured request window
- 262,144 tokens
- Longest tested prompt
- 245,760 target input tokens
- Validated retrieval
- ≈240K input tokens · passkey passed
Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.
| Workload (c1) | Input tok/s | TTFT p95 (s) |
|---|---|---|
| 4,096 input tokens | 560.42 | 2.84 |
| 32,768 input tokens | 1,096.57 | 28.40 |
| 131,072 input tokens | 1,252.93 | 100.93 |
| 245,760 input tokensPasskey prompt | 1,288.72 | 188.82 |
Sustained load
20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.
- Aggregate throughput
- 60.90 output tok/s
- TTFT p95
- 4.22 s
- Completed requests
- 144
- Request errors
- 0
Serving regression checks 13/13 passed
Tool-use evaluation
Unavailable · no published tool-eval result for this configuration.