GLM-5.3-Flash-EXL3 two-Spark serving baseline
2× DGX Spark · vLLM ·
Tested configuration
- Hardware
- 2x NVIDIA DGX Spark (128GB each)
- Serving runtime
- vLLM
- Runtime version or build
- sha256:9bb1557a4234fce63d59599e44d10747eabd742beb337eebf9e7070be8a0fd58
- Weight quantization or precision
- EXL3
- Benchmark date
- Recipe credit
- MiaAI-Lab original recipeBy MiaAI-Lab
79f10b91f84779b2b1ff2c9327b1a5847cd97f70
Decode sweep
~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.
| Concurrency | Output tok/s | TTFT p95 (s) |
|---|---|---|
| 1 | 35.13 | 0.95 |
| 2 | 40.52 | 15.78 |
| 4 | 62.52 | 23.65 |
| 8 | 70.02 | 50.09 |
| 12 | 76.97 | 65.28 |
| 16 | 79.51 | 91.26 |
Prefill and context
- Configured request window
- 1,000,000 tokens
- Longest tested prompt
- 245,760 target input tokens
- Validated retrieval
- ≈240K input tokens · passkey passed
Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.
| Workload (c1) | Input tok/s | TTFT p95 (s) |
|---|---|---|
| 4,096 input tokens | 566.73 | 5.00 |
| 32,768 input tokens | 833.16 | 36.85 |
| 131,072 input tokens | 882.52 | 147.00 |
| 245,760 input tokensPasskey prompt | 876.11 | 279.80 |
Sustained load
20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.
- Aggregate throughput
- 38.53 output tok/s
- TTFT p95
- 98.77 s
- Completed requests
- 98
- Request errors
- 0
Serving regression checks 13/13 passed
Tool-use evaluation
Unavailable · no published tool-eval result for this configuration.