Spark Bench

Qwen3.8-Flash-Next NVIDIA NVFP4 on TensorFold, two Sparks

2× DGX Spark · TensorFold ·

Tested configuration

Hardware
2x NVIDIA DGX Spark (128GB each)
Serving runtime
TensorFold
Runtime version or build
TensorFold Zig engine from commit db281878ddb836fd0df510d8771ecb7e0fe47d26, built with recipe patch set f3e1300f4f62 and Zig 0.17.0. · ghcr.io/miaai-lab/qwen3.8-flash-dual-dgx-sparks-tensorfold@sha256:e188128ddabdc67ac02c05781f5c0c6b348db5f050cac297861d82b7f086e18c; identical local image ID sha256:5eb5d4d384773543109baa26783fc6ea38ad619dff330f01672e05c01721cfb0 on both ranks.
Weight quantization or precision
NVFP4
Benchmark date
Recipe credit
MiaAI-Lab TensorFold recipeBy MiaAI-Labbe3ed857ab1055ca47b36134d22691f00f2bb94a

Decode sweep

~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.

ConcurrencyOutput tok/sTTFT p95 (s)
183.450.23
2129.240.43
4195.560.68
8282.691.02
12362.591.44
16417.632.11

Prefill and context

Configured request window
1,048,576 tokens
Longest tested prompt
245,760 target input tokens
Validated retrieval
≈240K input tokens · passkey passed

Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.

Workload (c1)Input tok/sTTFT p95 (s)
4,096 input tokens2,010.451.51
32,768 input tokens2,673.3611.24
131,072 input tokens2,614.7448.97
245,760 input tokensPasskey prompt2,436.0198.98

Sustained load

20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.

Aggregate throughput
432.90 output tok/s
TTFT p95
1.00 s
Completed requests
1,019
Request errors
0

Serving regression checks 13/13 passed

Tool-use evaluation

Incomplete pilot · 98.1% completionSafety gate passed

Comparisons require complete results with 100% completion under the same protocol ID. This run is retained as individual evidence.

Protocol ID
tool-eval-standard-v1-b60e2f0
Workload
69 mock-tool scenarios per trial · 3 trials
Mean score
88.0 / 100
Pass@3 · passed at least once
79.4%
Pass³ · passed every trial
79.4%

Read the tool-eval report and scenario findings

Other configurations for Qwen3.8-Flash-Next NVFP4