Spark Bench

Qwen3.8 27B MLX 4-bit on TensorFold, one Spark, thinking off

1× DGX Spark · TensorFold ·

Tested configuration

Hardware
1x NVIDIA DGX Spark (128GB)
Serving runtime
TensorFold
Runtime version or build
TensorFold v0.6.0 with the recipe's five patches (image tag tensorfold-qwen38-27b:v0.6.0, patch hash fb91a0019fdc), built locally on nvcr.io/nvidia/pytorch:26.07-py3
Weight quantization or precision
MLX 4-bit
Benchmark date
Recipe credit
MiaAI-Lab Qwen3.8 27B TensorFold recipeBy MiaAI-Labc11ff138871cf43d12898d79dbfa4dc952d17ca4

Decode sweep

~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.

ConcurrencyOutput tok/sTTFT p95 (s)
150.250.44
289.480.78
4128.821.38
8179.132.86
12197.6819.64
16201.4925.03

Prefill and context

Configured request window
262,144 tokens
Longest tested prompt
245,760 target input tokens
Validated retrieval
≈240K input tokens · passkey passed

Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.

Workload (c1)Input tok/sTTFT p95 (s)
4,096 input tokens1,015.322.48
32,768 input tokens1,478.4220.58
131,072 input tokens1,049.04122.39
245,760 input tokensPasskey prompt787.01311.47

Sustained load

20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.

Aggregate throughput
160.59 output tok/s
TTFT p95
2.35 s
Completed requests
382
Request errors
0

Serving regression checks 13/13 passed

Tool-use evaluation

Incomplete pilot · 98.6% completionSafety gate failed

Comparisons require complete results with 100% completion under the same protocol ID. This run is retained as individual evidence.

Protocol ID
tool-eval-standard-v1-b60e2f0
Workload
69 mock-tool scenarios per trial · 3 trials
Mean score
90.0 / 100
Pass@3 · passed at least once
85.3%
Pass³ · passed every trial
85.3%

Read the tool-eval report and scenario findings

Other configurations for Qwen3.8 27B