Nemotron 3.5 Super VL 120B-A12B NVFP4 on vLLM with DFlash, one Spark, thinking off
1× DGX Spark · vLLM ·
Tested configuration
- Hardware
- 1x NVIDIA DGX Spark (128GB)
- Serving runtime
- vLLM
- Runtime version or build
- vLLM 0.32.0rc1.dev190+g3f1bcfd82
- Weight quantization or precision
- NVFP4
- Benchmark date
- Recipe credit
- Kindling Nemotron 3.5 Super VL DGX Spark recipeBy kindlingai
452acc1d74e76cfee4b45ad7a16998ee57f0a423
Decode sweep
~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.
| Concurrency | Output tok/s | TTFT p95 (s) |
|---|---|---|
| 1 | 34.15 | 0.58 |
| 2 | 34.01 | 16.58 |
| 4 | 34.56 | 47.22 |
| 8 | 34.17 | 105.30 |
| 12 | 34.95 | 167.49 |
| 16 | 34.85 | 227.70 |
Prefill and context
- Configured request window
- 1,048,576 tokens
- Longest tested prompt
- 245,760 target input tokens
- Validated retrieval
- ≈240K input tokens · passkey passed
Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.
| Workload (c1) | Input tok/s | TTFT p95 (s) |
|---|---|---|
| 4,096 input tokens | 980.80 | 1.86 |
| 32,768 input tokens | 1,934.07 | 14.85 |
| 131,072 input tokens | 1,839.10 | 69.19 |
| 245,760 input tokensPasskey prompt | 1,577.68 | 155.41 |
Sustained load
20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.
- Aggregate throughput
- 33.90 output tok/s
- TTFT p95
- 113.03 s
- Completed requests
- 87
- Request errors
- 0
Serving regression checks 13/13 passed
Tool-use evaluation
Complete · 100% completionSafety gate failed
Compare only against complete results with 100% completion under the same protocol ID.
- Protocol ID
tool-eval-standard-v1-b60e2f0- Workload
- 69 mock-tool scenarios per trial · 3 trials
- Mean score
- 92.7 / 100
- Pass@3 · passed at least once
- 97.1%
- Pass³ · passed every trial
- 81.2%