GLM-5.3 Flash EXL3 TensorFold v0.6.0 two-Spark baseline
2× DGX Spark · TensorFold ·
Tested configuration
- Hardware
- 2x NVIDIA DGX Spark (128GB each)
- Serving runtime
- TensorFold
- Runtime version or build
- TensorFold v0.6.0 with patch hash ae8d1c789b47
- Weight quantization or precision
- EXL3
- Benchmark date
- Recipe credit
- MiaAI-Lab TensorFold recipeBy MiaAI-Lab
978b2252059069b3b4b84f0f7eeb73bc17f28d3f
Decode sweep
~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.
| Concurrency | Output tok/s | TTFT p95 (s) |
|---|---|---|
| 1 | 51.49 | 0.41 |
| 2 | 67.95 | 1.49 |
| 4 | 92.68 | 1.06 |
| 8 | 93.53 | 24.66 |
| 12 | 96.72 | 46.79 |
| 16 | 94.55 | 71.70 |
Prefill and context
- Configured request window
- 1,048,576 tokens
- Longest tested prompt
- 245,760 target input tokens
- Validated retrieval
- ≈240K input tokens · passkey passed
Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.
| Workload (c1) | Input tok/s | TTFT p95 (s) |
|---|---|---|
| 4,096 input tokens | 913.31 | 3.91 |
| 32,768 input tokens | 1,662.87 | 18.25 |
| 131,072 input tokens | 1,625.71 | 79.30 |
| 245,760 input tokensPasskey prompt | 1,484.02 | 163.89 |
Sustained load
20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.
- Aggregate throughput
- 84.90 output tok/s
- TTFT p95
- 28.42 s
- Completed requests
- 204
- Request errors
- 0
Serving regression checks 10/13 passed
Tool-use evaluation
Unavailable · no published tool-eval result for this configuration.