Qwen3.8 27B Uncensored FP8 single-Spark serving baseline
1× DGX Spark · SGLang ·
Tested configuration
- Hardware
- 1x NVIDIA DGX Spark (128GB)
- Serving runtime
- SGLang
- Runtime version or build
- SGLang container lmsysorg/sglang@sha256:febfb971c7352570fc445c466ebd6ffc9d896024958e544a60f2137fd85856b1
- Weight quantization or precision
- Uncensored FP8
- Benchmark date
- Recipe credit
- MiaAI-Lab Qwen3.8 27B SGLang recipeBy MiaAI-Lab
5d2df792a2ca7e076cb80b8302f5349d492d6f54
Decode sweep
~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.
| Concurrency | Output tok/s | TTFT p95 (s) |
|---|---|---|
| 1 | 18.60 | 0.28 |
| 2 | 36.82 | 0.47 |
| 4 | 72.86 | 0.84 |
| 8 | 69.59 | 31.51 |
| 12 | 68.50 | 67.37 |
| 16 | 70.81 | 88.06 |
Prefill and context
- Configured request window
- 262,144 tokens
- Longest tested prompt
- 245,760 target input tokens
- Validated retrieval
- ≈240K input tokens · passkey passed
Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.
| Workload (c1) | Input tok/s | TTFT p95 (s) |
|---|---|---|
| 4,096 input tokens | 547.32 | 3.44 |
| 32,768 input tokens | 710.55 | 41.82 |
| 131,072 input tokens | 563.45 | 228.68 |
| 245,760 input tokensPasskey prompt | 439.89 | 557.96 |
Sustained load
20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.
- Aggregate throughput
- 63.89 output tok/s
- TTFT p95
- 36.05 s
- Completed requests
- 157
- Request errors
- 0
Serving regression checks 13/13 passed
Tool-use evaluation
Unavailable · no published tool-eval result for this configuration.