Spark Bench

Qwen3.8 27B NVFP4 single-Spark serving baseline

1× DGX Spark · SGLang ·

Tested configuration

Hardware
1x NVIDIA DGX Spark (128GB)
Serving runtime
SGLang
Runtime version or build
SGLang container lmsysorg/sglang@sha256:febfb971c7352570fc445c466ebd6ffc9d896024958e544a60f2137fd85856b1
Weight quantization or precision
NVFP4
Benchmark date
Recipe credit
MiaAI-Lab Qwen3.8 27B SGLang recipeBy MiaAI-Lab5d2df792a2ca7e076cb80b8302f5349d492d6f54

Decode sweep

~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.

ConcurrencyOutput tok/sTTFT p95 (s)
131.830.18
257.030.42
498.930.78
8167.351.63
12184.1220.23
16187.6323.50

Prefill and context

Configured request window
262,144 tokens
Longest tested prompt
245,760 target input tokens
Validated retrieval
≈240K input tokens · passkey passed

Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.

Workload (c1)Input tok/sTTFT p95 (s)
4,096 input tokens968.661.90
32,768 input tokens1,402.9421.15
131,072 input tokens908.73141.58
245,760 input tokensPasskey prompt621.28395.05

Sustained load

20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.

Aggregate throughput
157.80 output tok/s
TTFT p95
1.01 s
Completed requests
377
Request errors
0

Serving regression checks 13/13 passed

Tool-use evaluation

Unavailable · no published tool-eval result for this configuration.

Other configurations for Qwen3.8 27B