Spark Bench

GLM-5.3-Flash-EXL3 two-Spark serving baseline

2× DGX Spark · vLLM ·

Tested configuration

Hardware
2x NVIDIA DGX Spark (128GB each)
Serving runtime
vLLM
Runtime version or build
sha256:9bb1557a4234fce63d59599e44d10747eabd742beb337eebf9e7070be8a0fd58
Weight quantization or precision
EXL3
Benchmark date
Recipe credit
MiaAI-Lab original recipeBy MiaAI-Lab79f10b91f84779b2b1ff2c9327b1a5847cd97f70

Decode sweep

~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.

ConcurrencyOutput tok/sTTFT p95 (s)
135.130.95
240.5215.78
462.5223.65
870.0250.09
1276.9765.28
1679.5191.26

Prefill and context

Configured request window
1,000,000 tokens
Longest tested prompt
245,760 target input tokens
Validated retrieval
≈240K input tokens · passkey passed

Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.

Workload (c1)Input tok/sTTFT p95 (s)
4,096 input tokens566.735.00
32,768 input tokens833.1636.85
131,072 input tokens882.52147.00
245,760 input tokensPasskey prompt876.11279.80

Sustained load

20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.

Aggregate throughput
38.53 output tok/s
TTFT p95
98.77 s
Completed requests
98
Request errors
0

Serving regression checks 13/13 passed

Tool-use evaluation

Unavailable · no published tool-eval result for this configuration.

Other configurations for GLM-5.3 Flash