Spark Bench

GLM-5.3 Flash EXL3 4 bpw: TensorFold v0.6.0 with the DFlash2 drafter, two Sparks, 1M (recipe 3fd60dfc)

2× DGX Spark · TensorFold ·

Tested configuration

Hardware
2x NVIDIA DGX Spark (128GB each)
Serving runtime
TensorFold
Runtime version or build
TensorFold v0.6.0 (commit c4646171) plus the recipe's patches, on nvcr.io/nvidia/pytorch:26.07-py3
Weight quantization or precision
EXL3 4 bpw
Benchmark date
Recipe credit
MiaAI-Lab TensorFold recipeBy MiaAI-Lab3fd60dfc836fab0f1ed0fd4f0afaf2a1d62b2aab

Decode sweep

~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.

ConcurrencyOutput tok/sTTFT p95 (s)
154.530.44
274.560.66
493.200.91
891.8624.45
1285.1452.92
1690.7674.50

Prefill and context

Configured request window
1,048,576 tokens
Longest tested prompt
131,072 target input tokens
Validated retrieval
UnavailablePasskey not run

Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.

Workload (c1)Input tok/sTTFT p95 (s)
4,096 input tokens882.504.34
32,768 input tokens1,744.9617.36
131,072 input tokens1,777.9274.18

Sustained load

20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.

Aggregate throughput
80.18 output tok/s
TTFT p95
29.98 s
Completed requests
194
Request errors
0

Serving regression checks 12/12 passed

Tool-use evaluation

Complete · 100% completionSafety gate failed

Compare only against complete results with 100% completion under the same protocol ID.

Protocol ID
tool-eval-standard-v1-b60e2f0
Workload
69 mock-tool scenarios per trial · 3 trials
Mean score
94.0 / 100
Pass@3 · passed at least once
92.8%
Pass³ · passed every trial
92.8%

Read the tool-eval report and scenario findings

Other configurations for GLM-5.3 Flash