Qwen3.8-Flash-Next NVIDIA NVFP4 on TensorFold, two Sparks
2× DGX Spark · TensorFold ·
Tested configuration
- Hardware
- 2x NVIDIA DGX Spark (128GB each)
- Serving runtime
- TensorFold
- Runtime version or build
- TensorFold Zig engine from commit db281878ddb836fd0df510d8771ecb7e0fe47d26, built with recipe patch set f3e1300f4f62 and Zig 0.17.0. · ghcr.io/miaai-lab/qwen3.8-flash-dual-dgx-sparks-tensorfold@sha256:e188128ddabdc67ac02c05781f5c0c6b348db5f050cac297861d82b7f086e18c; identical local image ID sha256:5eb5d4d384773543109baa26783fc6ea38ad619dff330f01672e05c01721cfb0 on both ranks.
- Weight quantization or precision
- NVFP4
- Benchmark date
- Recipe credit
- MiaAI-Lab TensorFold recipeBy MiaAI-Lab
be3ed857ab1055ca47b36134d22691f00f2bb94a
Decode sweep
~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.
| Concurrency | Output tok/s | TTFT p95 (s) |
|---|---|---|
| 1 | 83.45 | 0.23 |
| 2 | 129.24 | 0.43 |
| 4 | 195.56 | 0.68 |
| 8 | 282.69 | 1.02 |
| 12 | 362.59 | 1.44 |
| 16 | 417.63 | 2.11 |
Prefill and context
- Configured request window
- 1,048,576 tokens
- Longest tested prompt
- 245,760 target input tokens
- Validated retrieval
- ≈240K input tokens · passkey passed
Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.
| Workload (c1) | Input tok/s | TTFT p95 (s) |
|---|---|---|
| 4,096 input tokens | 2,010.45 | 1.51 |
| 32,768 input tokens | 2,673.36 | 11.24 |
| 131,072 input tokens | 2,614.74 | 48.97 |
| 245,760 input tokensPasskey prompt | 2,436.01 | 98.98 |
Sustained load
20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.
- Aggregate throughput
- 432.90 output tok/s
- TTFT p95
- 1.00 s
- Completed requests
- 1,019
- Request errors
- 0
Serving regression checks 13/13 passed
Tool-use evaluation
Incomplete pilot · 98.1% completionSafety gate passed
Comparisons require complete results with 100% completion under the same protocol ID. This run is retained as individual evidence.
- Protocol ID
tool-eval-standard-v1-b60e2f0- Workload
- 69 mock-tool scenarios per trial · 3 trials
- Mean score
- 88.0 / 100
- Pass@3 · passed at least once
- 79.4%
- Pass³ · passed every trial
- 79.4%