Spark Bench

GLM-5.3 Flash EXL3 4 bpw: TensorFold 1.3.0 with DFlash2, P4, two Sparks, 1M

2× DGX Spark · TensorFold ·

Tested configuration

Hardware
2x NVIDIA DGX Spark (128GB each)
Serving runtime
TensorFold
Runtime version or build
TensorFold 1.3.0 from 81de26f64967 plus the recipe's 16-patch stack
Weight quantization or precision
EXL3 4 bpw
Benchmark date
Recipe credit
MiaAI-Lab GLM-5.3 Flash 1104 recipeBy MiaAI-Laba691916c16b6287ebb79ef8ee37ea182b5240a29

Decode sweep

~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.

ConcurrencyOutput tok/sTTFT p95 (s)
141.850.51
254.700.86
471.691.63
877.0230.08
1271.9860.51
1673.0794.00

Prefill and context

Configured request window
1,048,576 tokens
Longest tested prompt
245,760 target input tokens
Validated retrieval
≈240K input tokens · passkey passed

Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.

Workload (c1)Input tok/sTTFT p95 (s)
4,096 input tokens772.713.63
32,768 input tokens1,170.3226.18
131,072 input tokens1,226.52105.73
245,760 input tokensPasskey prompt1,226.54199.29

Sustained load

20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.

Aggregate throughput
65.01 output tok/s
TTFT p95
37.75 s
Completed requests
158
Request errors
0

Serving regression checks 13/13 passed

Tool-use evaluation

Complete · 100% completionSafety gate failed

Compare only against complete results with 100% completion under the same protocol ID.

Protocol ID
tool-eval-standard-v1-b60e2f0
Workload
69 mock-tool scenarios per trial · 3 trials
Mean score
91.0 / 100
Pass@3 · passed at least once
88.4%
Pass³ · passed every trial
88.4%

Read the tool-eval report and scenario findings

Other configurations for GLM-5.3 Flash