DeepSeek V4.1 Flash 1017 EXL3-EAQ 2.9 bpw: TensorFold native, two Sparks (TP=2), 262K, recipe 27741e17, thinking off
2× DGX Spark · TensorFold ·
Tested configuration
- Hardware
- 2x NVIDIA DGX Spark (128GB each)
- Serving runtime
- TensorFold
- Runtime version or build
- TensorFold (native Zig, TP=2) built locally with the recipe's build arguments and PULL=0 (no registry image was pulled)
- Weight quantization or precision
- EXL3-EAQ 2.9 bpw
- Benchmark date
- Recipe credit
- MiaAI-Lab DeepSeek V4.1 Flash 1017 recipeBy MiaAI-Lab
27741e176a9e5b98c10c9c7d012fd3e4fcdc19b6
Decode sweep
~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.
| Concurrency | Output tok/s | TTFT p95 (s) |
|---|---|---|
| 1 | 67.50 | 0.51 |
| 2 | 93.97 | 0.64 |
| 4 | 124.87 | 1.24 |
| 8 | 124.19 | 18.91 |
| 12 | 124.33 | 35.64 |
| 16 | 125.87 | 51.85 |
Prefill and context
- Configured request window
- 262,144 tokens
- Longest tested prompt
- 245,760 target input tokens
- Validated retrieval
- ≈240K input tokens · passkey passed
Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.
| Workload (c1) | Input tok/s | TTFT p95 (s) |
|---|---|---|
| 4,096 input tokens | 1,298.28 | 2.20 |
| 32,768 input tokens | 2,008.07 | 15.27 |
| 131,072 input tokens | 2,008.21 | 64.30 |
| 245,760 input tokensPasskey prompt | 2,037.39 | 120.40 |
Sustained load
20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.
- Aggregate throughput
- 108.73 output tok/s
- TTFT p95
- 22.45 s
- Completed requests
- 261
- Request errors
- 0
Serving regression checks 13/13 passed
Tool-use evaluation
Incomplete pilot · 98.6% completionSafety gate passed
Comparisons require complete results with 100% completion under the same protocol ID. This run is retained as individual evidence.
- Protocol ID
tool-eval-standard-v1-b60e2f0- Workload
- 69 mock-tool scenarios per trial · 3 trials
- Mean score
- 90.0 / 100
- Pass@3 · passed at least once
- 82.4%
- Pass³ · passed every trial
- 82.4%