Spark Bench

NVIDIA Nemotron 3.5 Bolt 74B-A9B on TensorFold, one Spark

1× DGX Spark · TensorFold ·

Tested configuration

Hardware
1x NVIDIA DGX Spark (128GB)
Serving runtime
TensorFold
Runtime version or build
tensorfold-native with the recipe's Bolt lane backend (libbolt), built locally from the recipe's pinned TensorFold reference and patch set
Weight quantization or precision
IQ2_XXS experts, NVFP4 Mamba projections, and Q8_0 attention, latent, and head projections
Benchmark date
Recipe credit
MiaAI-Lab Nemotron 3.5 Bolt TensorFold recipeBy MiaAI-Lab6f61d7727ae3fac010f30e240799cc884d295f6e

Decode sweep

~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.

ConcurrencyOutput tok/sTTFT p95 (s)
179.230.44
2106.590.44
4149.490.64
8180.560.92
12205.055.12
16234.452.49

Prefill and context

Configured request window
262,144 tokens
Longest tested prompt
245,760 target input tokens
Validated retrieval
≈240K input tokens · passkey passed

Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.

Workload (c1)Input tok/sTTFT p95 (s)
4,096 input tokens1,150.462.70
32,768 input tokens1,421.4922.09
131,072 input tokens1,076.89120.84
245,760 input tokensPasskey prompt829.93294.65

Sustained load

20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.

Aggregate throughput
145.52 output tok/s
TTFT p95
2.30 s
Completed requests
345
Request errors
0

Serving regression checks 13/13 passed

Tool-use evaluation

Complete · 100% completionSafety gate failed

Compare only against complete results with 100% completion under the same protocol ID.

Protocol ID
tool-eval-standard-v1-b60e2f0
Workload
69 mock-tool scenarios per trial · 3 trials
Mean score
90.3 / 100
Pass@3 · passed at least once
85.5%
Pass³ · passed every trial
85.5%

Read the tool-eval report and scenario findings