Spark Bench

GLM-5.3 Flash EXL3/TR3 4 bpw refreshed two-Spark baseline

2× DGX Spark · vLLM ·

Tested configuration

Hardware
2x NVIDIA DGX Spark (128GB each)
Serving runtime
vLLM
Runtime version or build
vLLM 0.1.dev20051+g487ecf187
Weight quantization or precision
EXL3/TR3 4 bpw
Benchmark date
Recipe credit
MiaAI-Lab refreshed GLM recipeBy MiaAI-Laba35eaab128233d215b2fc9ac261eddbad23c946d

Decode sweep

~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.

ConcurrencyOutput tok/sTTFT p95 (s)
113.961.27
222.6815.26
430.6810.48
825.60125.72
1227.61173.89
1626.88267.50

Prefill and context

Configured request window
850,000 tokens
Longest tested prompt
245,760 target input tokens
Validated retrieval
≈240K input tokens · passkey passed

Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.

Workload (c1)Input tok/sTTFT p95 (s)
4,096 input tokens420.555.27
32,768 input tokens781.3037.52
131,072 input tokens873.52147.68
245,760 input tokensPasskey prompt843.89287.63

Sustained load

20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.

Aggregate throughput
21.88 output tok/s
TTFT p95
181.03 s
Completed requests
56
Request errors
0

Serving regression checks 13/13 passed

Tool-use evaluation

Unavailable · no published tool-eval result for this configuration.

Other configurations for GLM-5.3 Flash