Spark Bench

NousResearch Arete-27B on SGLang, one Spark, FP8 KV, 262K

1× DGX Spark · SGLang ·

Tested configuration

Hardware
1x NVIDIA DGX Spark (128GB)
Serving runtime
SGLang
Runtime version or build
SGLang image lmsysorg/sglang@sha256:febfb971c7352570fc445c466ebd6ffc9d896024958e544a60f2137fd85856b1.
Weight quantization or precision
BF16
Benchmark date
Recipe credit
NousResearch/Arete-27B model cardBy NousResearchNousResearch/Arete-27B model card; local pinned SGLang launch configuration

Decode sweep

~256 input / 512 output tokens per request. Output throughput is aggregate across concurrent requests. TTFT means time to first token.

ConcurrencyOutput tok/sTTFT p95 (s)
14.290.61
28.630.88
416.951.14
832.422.34
1232.61130.35
1632.65130.41

Prefill and context

Configured request window
262,144 tokens
Longest tested prompt
245,760 target input tokens
Validated retrieval
≈240K input tokens · passkey passed

Input throughput uses full request wall time. The passkey measures retrieval from one long prompt.

Workload (c1)Input tok/sTTFT p95 (s)
4,096 input tokens225.163.48
32,768 input tokens719.6930.60
131,072 input tokens683.41175.95
245,760 input tokensPasskey prompt539.69452.84

Sustained load

20-minute soak at concurrency 8 · ~1024 input / 512 output tokens per request.

Aggregate throughput
31.28 output tok/s
TTFT p95
6.84 s
Completed requests
80
Request errors
0

Serving regression checks 13/13 passed

Tool-use evaluation

Complete · 100% completionSafety gate passed

Compare only against complete results with 100% completion under the same protocol ID.

Protocol ID
tool-eval-standard-v1-b60e2f0
Workload
69 mock-tool scenarios per trial · 3 trials
Mean score
95.0 / 100
Pass@3 · passed at least once
91.3%
Pass³ · passed every trial
91.3%

Read the tool-eval report and scenario findings