Benchmarks

Measured prefill and decode speed, with the hardware and the source. These are not our benchmarks: they are the best published measurements on M5-class hardware, plus Windows, Linux and CUDA references. Our Bosgame M5s have not arrived yet; StrixFold’s third-party pair data is below.

Measured rows
13
published sources
Bosgame M5 pair
58.2
decode tok/s; 1,284 prefill
Our M5
Pending
delivery Monday; plan below

StrixFold measured two Bosgame M5 pairs and a pair of Ryzen AI Halo boxes over USB4. TensorFold measured a single Strix Halo on ROCm. None of these are our boxes; our M5s arrive Monday. The site's token/s figures are theoretical ceilings; this page is the measured counterpart.

Measured

StrixFold is the two-box HIP fork of TensorFold; the M5 pair rows are third-party measurements on actual Bosgame M5s. TensorFold rows are its ROCm benchmark receipts on a single Strix Halo.

Updated 2026-11-21. Prefill is prompt processing speed; decode is token generation speed; TTFT is time to first token. Context matters, so the context column is part of the result.

Measured benchmark results with hardware and source
ModelEngineContextPrefill (tok/s)Decode (tok/s)TTFTSource
Qwen3.8 Flash-Next IQ3_S (Strata) measured
GMKtec NucBox EVO-X2 · Ryzen AI Max+ 395, 128 GB, Radeon 8060S · Windows 11 Pro
Third-party measurement on the closest M5-class box, not a Bosgame. 4K/16K/32K prompt reads were 755/821/792 tok/s; 4K TTFT 0.60 s.
Strata 0.2.7 / ROCm 10.2short chat75544.00.60 ssource
Qwen3.8 Flash-Next IQ3_S (Strata) measured
GMKtec NucBox EVO-X2 · Ryzen AI Max+ 395, 128 GB, Radeon 8060S · Windows 11 Pro
69% of the prompt came from the conversation cache; 4.5 s to first token per turn.
Strata 0.2.7 / ROCm 10.2coding agent, 6 turnsn/a47.04.50 ssource
Qwen3.8 Flash-Next IQ3_S (Strata) measured
Ryzen AI Max+ 395 test box · 128 GB, Radeon 8060S, Linux · Linux
Tuned switches. The same page measured 56.2 tok/s at 8K with defaults and 40.6-42.8 tok/s at 128K.
Strata 0.2.x8Kn/a59.7n/asource
GLM-5.3-Flash Maya-S measured
Radeon 8060S test box · Ryzen AI Max+ 395, 128 GB, 96 GB for the GPU · Windows 11
A user report for Maya-S, not the archived Maya-M. Short-context generation was 18.3 tok/s; prompt read 320 tok/s.
Project Maya 1.0.30Maya-S32018.3n/asource
Qwen3.8-27B-S Mirai measured
RTX 4070 · RTX 4070, 12 GB · Linux
CUDA reference, not an M5 result. 60.9 tok/s at 64K, 20.3 at 128K, 10.3 at 200K; 1,090 tok/s prompt read.
mirai-s-ada / CUDA 124K109070.1n/asource
StrixFold · DeepSeek V4.1 Flash 1017 measured
Bosgame M5 pair (third-party) · 2× Bosgame M5: Ryzen AI Max+ 395, 128 GB each, 2× USB4 · Linux, kernel 7.2.7, ROCm 10
Code 70.7 tok/s and prose 41.1 tok/s in the first-light test; one chat across the prompt set 58.2 tok/s; prompt read 1,284 tok/s. Stock 85 W balanced, drafting on.
StrixFold HIP / EXL3 2.9 bpwfirst light, one chat128458.2n/asource
StrixFold · DeepSeek V4.1 Flash 1017 measured
Bosgame M5 pair A (third-party) · 2× Bosgame M5: Ryzen AI Max+ 395, 128 GB each, USB4 · Linux, kernel 7.2.7, ROCm 10
Median of 47 single requests; 20-150K prompts ranged 48.3-51.6 tok/s. 27.3 tok/s each with the prompt cache; stock balanced 85 W, up to four concurrent.
StrixFold HIP / EXL3 2.9 bpwagent work, <20K prompt112854.5n/asource
StrixFold · DeepSeek V4.1 Flash 1017 measured
Bosgame M5 pair B (third-party) · 2× Bosgame M5: Ryzen AI Max+ 395, 128 GB each, USB4 · Linux, kernel 7.2.7, ROCm 10
Median of 37 single requests; 20-100K prompts ranged 39.7-52.1 tok/s; sample size is small and the USB4 links changed. 27.6 tok/s each with the prompt cache.
StrixFold HIP / EXL3 2.9 bpwagent work, <20K prompt111849.7n/asource
StrixFold · DeepSeek V4.1 Flash 1017 measured
2× Ryzen AI Halo (third-party) · 2× Ryzen AI Max+ 395, 128 GB each, 2× USB4 · Linux, kernel 7.2.7, ROCm 10
1,543.4 tok/s prompt read at 11,594 tokens; eight chats total 107.96 tok/s; 72.3% of 2× DGX Spark. Quality MMLU-Pro 84.8%, HumanEval+ 92%.
StrixFold HIP / EXL3 2.9 bpwone chat141452.8n/asource
StrixFold · DeepSeek V4.1 Flash 1017 measured
2× Ryzen AI Halo (third-party) · 2× Ryzen AI Max+ 395, 128 GB each, 2× USB4 · Linux, kernel 7.2.7, ROCm 10
Drafting on; 31.5 tok/s with drafting off. 32.7 at the task alone; 1M prompt read was 1,071 tok/s (962 s) and a cached second turn returned in 3.9 s.
StrixFold HIP / EXL3 2.9 bpw48K prompt, coding137973.7n/asource
TensorFold · Qwen3.8-27B MLX-4bit measured
Strix Halo (Radeon 8060S) · Radeon 8060S, gfx1151; board and RAM not named · Linux, ROCm
Stream delivery tok/s. Chat T0 13.17, code T1 12.89, chat T1 12.90; first text 0.06-0.50 s. Local checkpoint receipts; not yet publishable on the board. Prefill is not reported in the receipt table.
TensorFold 1.0.12 HIPcode T0, community-v1n/a12.9n/asource
TensorFold · Qwen3.6-35B-A3B-MTP MLX-4bit measured
Strix Halo (Radeon 8060S) · Radeon 8060S, gfx1151; board and RAM not named · Linux, ROCm
Stream delivery tok/s. Chat T0 79.84, code T1 80.75, chat T1 72.96; first text 0.06-0.50 s. Local checkpoint receipts. Prefill is not reported in the receipt table.
TensorFold 1.0.12 HIPcode T0, community-v1n/a103.4n/asource
TensorFold · Qwen3.5-9B MLX-4bit measured
Strix Halo (Radeon 8060S) · Radeon 8060S, gfx1151; board and RAM not named · Linux, ROCm
Stream delivery tok/s. Chat T0 38.95, code T1 38.61, chat T1 38.51; first text 0.06-0.50 s. Local checkpoint receipts. Prefill is not reported in the receipt table.
TensorFold 1.0.12 HIPcode T0, community-v1n/a38.8n/asource

Method: Measured results from published sources, not our own benchmarks. decode_tok_s is token generation speed; prefill_tok_s is prompt processing; ttft_s is time to first token. Context matters: a result at 4K is not a result at 128K. A missing prefill cell means the source did not report it, not that it is zero. Machine-readable: data/benchmarks.json.

Prefill and decode

Prefill is how fast the model reads the prompt; it sets the time to first token and the cost of a long context. Decode is how fast it writes the answer; it sets sustained output. They are separate measurements, and a single “tokens per second” number without a label is ambiguous.

StrixFold reports both, so its M5 rows have both columns. TensorFold’s issue #1001 reports stream-delivery decode only; its prefill cells are n/a because the source did not measure them. That is a missing measurement, not a zero. The same applies to the coding and 8K rows from Strata.

Measured vs estimated

A green Fits on the fleet matrix is a memory-fit verdict, not a speed claim. The ≈ tok/s ceiling under it is bandwidth divided by active weight bytes. A model can fit and still be unpleasant at 4 tok/s; the 27B BF16 row is the obvious example. The measured table above is what happened on real hardware.

What is still missing

Our Bosgame M5 x1No one-box result from our units yet. StrixFold measured pairs only; a single-box test is still needed.
Our Bosgame M5 x2, RPCStrixFold has third-party pair results below; our own pair, RPC split and load time are still pending.
MacBook Pro M4 Max / Mac mini M4No clean local result yet. The same test should run on Metal before the M5 arrives.
Full 262K contextNo measured result at the full Qwen3.8 window; the KV cache and speed both change with depth.
Image, video and TTSThe page measures LLM token speed. Diffusion steps, video frames and real-time speech need their own benchmarks.

The Monday benchmark plan

Run the same commands on both boxes, at the same context, with the same build. Record the command, the model revision, the driver or ROCm version and the room temperature; a number without those is not a benchmark.

# One M5, after the one-box set is copied
llama-bench -m /models/official/unsloth-Qwen3.8-Flash-Next-UD-IQ4_XS/UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00004.gguf -p 512 -n 128 -ngl 99 -r 3

# Same build, longer prompt; this is where KV cache and prefill show up
llama-bench -m /models/.../Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00004.gguf -p 8192 -n 128 -ngl 99 -r 3

# Two boxes, after rpc-server is running on box B
rpc-server -H 192.168.12.72 -p 50052
llama-bench -m /models/.../DeepSeek-V4.1-Flash-1017-GSQ-RCO-IQ1_M-00001-of-00006.gguf --rpc 192.168.12.72:50052 -p 512 -n 128 -ngl 99 -r 2

Measure load time separately. The first token after a 16K prompt is not the same as steady-state decode, and a two-box result should report the link and the split.

What this page is not