Benchmarks
Measured prefill and decode speed, with the hardware and the source. These are not our benchmarks: they are the best published measurements on M5-class hardware, plus Windows, Linux and CUDA references. Our Bosgame M5s have not arrived yet; StrixFold’s third-party pair data is below.
StrixFold measured two Bosgame M5 pairs and a pair of Ryzen AI Halo boxes over USB4. TensorFold measured a single Strix Halo on ROCm. None of these are our boxes; our M5s arrive Monday. The site's token/s figures are theoretical ceilings; this page is the measured counterpart.
Measured
StrixFold is the two-box HIP fork of TensorFold; the M5 pair rows are third-party measurements on actual Bosgame M5s. TensorFold rows are its ROCm benchmark receipts on a single Strix Halo.
Updated 2026-11-21. Prefill is prompt processing speed; decode is token generation speed; TTFT is time to first token. Context matters, so the context column is part of the result.
| Model | Engine | Context | Prefill (tok/s) | Decode (tok/s) | TTFT | Source |
|---|---|---|---|---|---|---|
| Qwen3.8 Flash-Next IQ3_S (Strata) measured GMKtec NucBox EVO-X2 · Ryzen AI Max+ 395, 128 GB, Radeon 8060S · Windows 11 Pro Third-party measurement on the closest M5-class box, not a Bosgame. 4K/16K/32K prompt reads were 755/821/792 tok/s; 4K TTFT 0.60 s. | Strata 0.2.7 / ROCm 10.2 | short chat | 755 | 44.0 | 0.60 s | source |
| Qwen3.8 Flash-Next IQ3_S (Strata) measured GMKtec NucBox EVO-X2 · Ryzen AI Max+ 395, 128 GB, Radeon 8060S · Windows 11 Pro 69% of the prompt came from the conversation cache; 4.5 s to first token per turn. | Strata 0.2.7 / ROCm 10.2 | coding agent, 6 turns | n/a | 47.0 | 4.50 s | source |
| Qwen3.8 Flash-Next IQ3_S (Strata) measured Ryzen AI Max+ 395 test box · 128 GB, Radeon 8060S, Linux · Linux Tuned switches. The same page measured 56.2 tok/s at 8K with defaults and 40.6-42.8 tok/s at 128K. | Strata 0.2.x | 8K | n/a | 59.7 | n/a | source |
| GLM-5.3-Flash Maya-S measured Radeon 8060S test box · Ryzen AI Max+ 395, 128 GB, 96 GB for the GPU · Windows 11 A user report for Maya-S, not the archived Maya-M. Short-context generation was 18.3 tok/s; prompt read 320 tok/s. | Project Maya 1.0.30 | Maya-S | 320 | 18.3 | n/a | source |
| Qwen3.8-27B-S Mirai measured RTX 4070 · RTX 4070, 12 GB · Linux CUDA reference, not an M5 result. 60.9 tok/s at 64K, 20.3 at 128K, 10.3 at 200K; 1,090 tok/s prompt read. | mirai-s-ada / CUDA 12 | 4K | 1090 | 70.1 | n/a | source |
| StrixFold · DeepSeek V4.1 Flash 1017 measured Bosgame M5 pair (third-party) · 2× Bosgame M5: Ryzen AI Max+ 395, 128 GB each, 2× USB4 · Linux, kernel 7.2.7, ROCm 10 Code 70.7 tok/s and prose 41.1 tok/s in the first-light test; one chat across the prompt set 58.2 tok/s; prompt read 1,284 tok/s. Stock 85 W balanced, drafting on. | StrixFold HIP / EXL3 2.9 bpw | first light, one chat | 1284 | 58.2 | n/a | source |
| StrixFold · DeepSeek V4.1 Flash 1017 measured Bosgame M5 pair A (third-party) · 2× Bosgame M5: Ryzen AI Max+ 395, 128 GB each, USB4 · Linux, kernel 7.2.7, ROCm 10 Median of 47 single requests; 20-150K prompts ranged 48.3-51.6 tok/s. 27.3 tok/s each with the prompt cache; stock balanced 85 W, up to four concurrent. | StrixFold HIP / EXL3 2.9 bpw | agent work, <20K prompt | 1128 | 54.5 | n/a | source |
| StrixFold · DeepSeek V4.1 Flash 1017 measured Bosgame M5 pair B (third-party) · 2× Bosgame M5: Ryzen AI Max+ 395, 128 GB each, USB4 · Linux, kernel 7.2.7, ROCm 10 Median of 37 single requests; 20-100K prompts ranged 39.7-52.1 tok/s; sample size is small and the USB4 links changed. 27.6 tok/s each with the prompt cache. | StrixFold HIP / EXL3 2.9 bpw | agent work, <20K prompt | 1118 | 49.7 | n/a | source |
| StrixFold · DeepSeek V4.1 Flash 1017 measured 2× Ryzen AI Halo (third-party) · 2× Ryzen AI Max+ 395, 128 GB each, 2× USB4 · Linux, kernel 7.2.7, ROCm 10 1,543.4 tok/s prompt read at 11,594 tokens; eight chats total 107.96 tok/s; 72.3% of 2× DGX Spark. Quality MMLU-Pro 84.8%, HumanEval+ 92%. | StrixFold HIP / EXL3 2.9 bpw | one chat | 1414 | 52.8 | n/a | source |
| StrixFold · DeepSeek V4.1 Flash 1017 measured 2× Ryzen AI Halo (third-party) · 2× Ryzen AI Max+ 395, 128 GB each, 2× USB4 · Linux, kernel 7.2.7, ROCm 10 Drafting on; 31.5 tok/s with drafting off. 32.7 at the task alone; 1M prompt read was 1,071 tok/s (962 s) and a cached second turn returned in 3.9 s. | StrixFold HIP / EXL3 2.9 bpw | 48K prompt, coding | 1379 | 73.7 | n/a | source |
| TensorFold · Qwen3.8-27B MLX-4bit measured Strix Halo (Radeon 8060S) · Radeon 8060S, gfx1151; board and RAM not named · Linux, ROCm Stream delivery tok/s. Chat T0 13.17, code T1 12.89, chat T1 12.90; first text 0.06-0.50 s. Local checkpoint receipts; not yet publishable on the board. Prefill is not reported in the receipt table. | TensorFold 1.0.12 HIP | code T0, community-v1 | n/a | 12.9 | n/a | source |
| TensorFold · Qwen3.6-35B-A3B-MTP MLX-4bit measured Strix Halo (Radeon 8060S) · Radeon 8060S, gfx1151; board and RAM not named · Linux, ROCm Stream delivery tok/s. Chat T0 79.84, code T1 80.75, chat T1 72.96; first text 0.06-0.50 s. Local checkpoint receipts. Prefill is not reported in the receipt table. | TensorFold 1.0.12 HIP | code T0, community-v1 | n/a | 103.4 | n/a | source |
| TensorFold · Qwen3.5-9B MLX-4bit measured Strix Halo (Radeon 8060S) · Radeon 8060S, gfx1151; board and RAM not named · Linux, ROCm Stream delivery tok/s. Chat T0 38.95, code T1 38.61, chat T1 38.51; first text 0.06-0.50 s. Local checkpoint receipts. Prefill is not reported in the receipt table. | TensorFold 1.0.12 HIP | code T0, community-v1 | n/a | 38.8 | n/a | source |
Method: Measured results from published sources, not our own benchmarks. decode_tok_s is token generation speed; prefill_tok_s is prompt processing; ttft_s is time to first token. Context matters: a result at 4K is not a result at 128K. A missing prefill cell means the source did not report it, not that it is zero. Machine-readable: data/benchmarks.json.
Prefill and decode
Prefill is how fast the model reads the prompt; it sets the time to first token and the cost of a long context. Decode is how fast it writes the answer; it sets sustained output. They are separate measurements, and a single “tokens per second” number without a label is ambiguous.
StrixFold reports both, so its M5 rows have both columns. TensorFold’s issue #1001 reports stream-delivery decode only; its prefill cells are n/a because the source did not measure them. That is a missing measurement, not a zero. The same applies to the coding and 8K rows from Strata.
Measured vs estimated
A green Fits on the fleet matrix is a memory-fit verdict, not a speed claim. The ≈ tok/s ceiling under it is bandwidth divided by active weight bytes. A model can fit and still be unpleasant at 4 tok/s; the 27B BF16 row is the obvious example. The measured table above is what happened on real hardware.
What is still missing
The Monday benchmark plan
Run the same commands on both boxes, at the same context, with the same build. Record the command, the model revision, the driver or ROCm version and the room temperature; a number without those is not a benchmark.
# One M5, after the one-box set is copied
llama-bench -m /models/official/unsloth-Qwen3.8-Flash-Next-UD-IQ4_XS/UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00004.gguf -p 512 -n 128 -ngl 99 -r 3
# Same build, longer prompt; this is where KV cache and prefill show up
llama-bench -m /models/.../Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00004.gguf -p 8192 -n 128 -ngl 99 -r 3
# Two boxes, after rpc-server is running on box B
rpc-server -H 192.168.12.72 -p 50052
llama-bench -m /models/.../DeepSeek-V4.1-Flash-1017-GSQ-RCO-IQ1_M-00001-of-00006.gguf --rpc 192.168.12.72:50052 -p 512 -n 128 -ngl 99 -r 2
Measure load time separately. The first token after a 16K prompt is not the same as steady-state decode, and a two-box result should report the link and the split.
What this page is not
- Not our benchmark: every row links to the person or project that measured it.
- Not a quality score. Speed says nothing about answer quality.
- Not a comparison across different prompts. The context column is part of the number.
- Not final. The M5 rows replace the closest-hardware rows once the boxes are here.