Spark Bench

GLM-5.3 Flash EXL3 TensorFold v0.6.0 two-Spark baseline

2× DGX Spark · TensorFold ·

Tested configuration

Hardware
2x NVIDIA DGX Spark (128GB each)
Serving runtime
TensorFold
Runtime version or build
TensorFold v0.6.0 with patch hash ae8d1c789b47
Weight quantization or precision
EXL3
Benchmark date
Recipe credit
MiaAI-Lab TensorFold recipeBy MiaAI-Lab978b2252059069b3b4b84f0f7eeb73bc17f28d3f

Decode sweep

~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.

ConcurrencyOutput tok/sTTFT p95 (s)
151.490.41
267.951.49
492.681.06
893.5324.66
1296.7246.79
1694.5571.70

Prefill and context

Configured request window
1,048,576 tokens
Longest tested prompt
245,760 target input tokens
Validated retrieval
≈240K input tokens · passkey passed

Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.

Workload (c1)Input tok/sTTFT p95 (s)
4,096 input tokens913.313.91
32,768 input tokens1,662.8718.25
131,072 input tokens1,625.7179.30
245,760 input tokensPasskey prompt1,484.02163.89

Sustained load

20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.

Aggregate throughput
84.90 output tok/s
TTFT p95
28.42 s
Completed requests
204
Request errors
0

Serving regression checks 10/13 passed

Tool-use evaluation

Unavailable · no published tool-eval result for this configuration.

Other configurations for GLM-5.3 Flash