Qwen3.8-Flash-Next FP4-Local (NVIDIA QAD) on TensorFold, one Spark, thinking off
1× DGX Spark · TensorFold ·
Tested configuration
- Hardware
- 1x NVIDIA DGX Spark (128GB)
- Serving runtime
- TensorFold
- Runtime version or build
- TensorFold db281878ddb8 (zig-flashnext) with the recipe's patches 0001-0015 (patch hash c0a2df4c680d). It was built locally with PULL=0 on nvcr.io/nvidia/pytorch:26.07-py3 and the recipe's sha256-checked Zig 0.17.0 toolchain, as image tag tensorfold-qwen38:zig-db28187
- Weight quantization or precision
- FP4-Local
- Benchmark date
- Recipe credit
- MiaAI-Lab Qwen3.8 Flash Next single-Spark TensorFold recipeBy MiaAI-Lab
656879fd752749bbd11a6e1a065a326c00940911
Decode sweep
~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.
| Concurrency | Output tok/s | TTFT p95 (s) |
|---|---|---|
| 1 | 34.29 | 0.48 |
| 2 | 57.62 | 0.64 |
| 4 | 75.71 | 0.51 |
| 8 | 89.30 | 20.46 |
| 12 | 100.43 | 41.47 |
| 16 | 95.69 | 61.00 |
Prefill and context
- Configured request window
- 262,144 tokens
- Longest tested prompt
- 245,760 target input tokens
- Validated retrieval
- ≈240K input tokens · passkey passed
Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.
| Workload (c1) | Input tok/s | TTFT p95 (s) |
|---|---|---|
| 4,096 input tokens | 1,176.04 | 1.95 |
| 32,768 input tokens | 1,962.37 | 15.03 |
| 131,072 input tokens | 1,956.29 | 65.10 |
| 245,760 input tokensPasskey prompt | 1,827.36 | 133.79 |
Sustained load
20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.
- Aggregate throughput
- 85.72 output tok/s
- TTFT p95
- 28.44 s
- Completed requests
- 207
- Request errors
- 0
Serving regression checks 13/13 passed
Tool-use evaluation
Incomplete pilot · 98.6% completionSafety gate passed
Comparisons require complete results with 100% completion under the same protocol ID. This run is retained as individual evidence.
- Protocol ID
tool-eval-standard-v1-b60e2f0- Workload
- 69 mock-tool scenarios per trial · 3 trials
- Mean score
- 86.0 / 100
- Pass@3 · passed at least once
- 76.5%
- Pass³ · passed every trial
- 76.5%