Recursive Flint 40B-A2.5B NVFP4 GGUF: llama.cpp b11856, one slot (-np 1), one Spark, 262K
1× DGX Spark · llama.cpp ·
Tested configuration
- Hardware
- 1x NVIDIA DGX Spark (128GB)
- Serving runtime
- llama.cpp
- Runtime version or build
- llama.cpp b11856 (3642238cc120, MIT), built locally from .devops/cuda.Dockerfile with CUDA 13.2 and CUDA architecture 121 (built as 121a). Nothing was pulled from a third-party registry and nothing was published
- Weight quantization or precision
- NVFP4 GGUF
- Benchmark date
- Recipe credit
- RecursiveSI/flint-nvfp4-ggufBy RecursiveSIModel card llama-server command (RecursiveSI/flint-nvfp4-gguf); no recipe repository
Decode sweep
~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.
| Concurrency | Output tok/s | TTFT p95 (s) |
|---|---|---|
| 1 | 81.73 | 0.32 |
| 2 | 83.40 | 6.46 |
| 4 | 82.85 | 18.95 |
| 8 | 83.03 | 43.81 |
| 12 | 81.67 | 69.43 |
| 16 | 82.95 | 93.22 |
Prefill and context
- Configured request window
- 262,144 tokens
- Longest tested prompt
- 245,760 target input tokens
- Validated retrieval
- ≈240K input tokens · passkey passed
Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.
| Workload (c1) | Input tok/s | TTFT p95 (s) |
|---|---|---|
| 4,096 input tokens | 1,485.24 | 2.00 |
| 32,768 input tokens | 1,848.59 | 16.74 |
| 131,072 input tokens | 1,436.04 | 89.89 |
| 245,760 input tokensPasskey prompt | 1,090.53 | 220.19 |
Sustained load
20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.
- Aggregate throughput
- 74.62 output tok/s
- TTFT p95
- 48.81 s
- Completed requests
- 182
- Request errors
- 0
Serving regression checks 10/13 passed
Tool-use evaluation
Complete · 100% completionSafety gate passed
Compare only against complete results with 100% completion under the same protocol ID.
- Protocol ID
tool-eval-standard-v1-b60e2f0- Workload
- 69 mock-tool scenarios per trial · 3 trials
- Mean score
- 86.0 / 100
- Pass@3 · passed at least once
- 84.1%
- Pass³ · passed every trial
- 75.4%