Spark Bench

GLM-5.3 Flash EXL3 4 bpw: TensorFold 1.1.9 with DFlash2, two Sparks, 1M

2× DGX Spark · TensorFold ·

Tested configuration

Hardware
2x NVIDIA DGX Spark (128GB each)
Serving runtime
TensorFold
Runtime version or build
TensorFold 1.1.9 from f350cd3e plus the recipe's 110 patches
Weight quantization or precision
EXL3 4 bpw
Benchmark date
Recipe credit
MiaAI-Lab GLM-5.3 Flash 1104 recipeBy MiaAI-Lab13f2464ee84ab2b5dabf0bb264d1144b02cf5fb7

Decode sweep

~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.

ConcurrencyOutput tok/sTTFT p95 (s)
142.670.37
258.010.72
479.041.19
8102.962.75
12114.574.06
16121.045.23

Prefill and context

Configured request window
1,048,576 tokens
Longest tested prompt
245,760 target input tokens
Validated retrieval
≈240K input tokens · passkey passed

Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.

Workload (c1)Input tok/sTTFT p95 (s)
4,096 input tokens902.982.65
32,768 input tokens1,569.7919.09
131,072 input tokens1,665.9777.03
245,760 input tokensPasskey prompt622.77391.26

Sustained load

20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.

Aggregate throughput
62.29 output tok/s
TTFT p95
38.12 s
Completed requests
149
Request errors
0

Serving regression checks 13/13 passed

Tool-use evaluation

Complete · 100% completionSafety gate failed

Compare only against complete results with 100% completion under the same protocol ID.

Protocol ID
tool-eval-standard-v1-b60e2f0
Workload
69 mock-tool scenarios per trial · 3 trials
Mean score
93.0 / 100
Pass@3 · passed at least once
91.3%
Pass³ · passed every trial
91.3%

Read the tool-eval report and scenario findings

Other configurations for GLM-5.3 Flash