Every model we keep, and where it runs.
Remnant Security's local AI archive: 28 model builds across text, vision, speech, music, image and video, about 2.9 TiB on disk. For each one: what it is, its license, how much memory it needs, and whether it runs on our machines, including the planned Bosgame M5 boxes, one or two of them.
Bosgame M5: one box vs two
One M5 is a Ryzen AI Max+ 395 with 128 GB of unified LPDDR5X at about 210-220 GB/s measured. On Linux about 108-110 GB is usable by the GPU (96 GB cap on Windows). Two boxes linked with llama.cpp RPC give roughly 200-225 GB, which adds capacity, not speed. Mixture-of-experts models are the sweet spot because only the active experts are read per token.
| Ready-to-run build | Abliterated | Est. memory | M5 ×1 | M5 ×2 | MacBook Pro M4 Max |
|---|---|---|---|---|---|
| DeepSeek V4.1 Flash 1017 GSQ-RCO IQ1_M backbone + shared Engram IQ4_NL + mmproj | no | ~130.8 GB | No | Fits | No |
| DeepSeek V4.1 Flash 1017 UD-IQ1_M GGUF (5 shards) + mmproj | yes | ~267.9 GB | No | No | No |
| GLM-5.3-Flash UD-IQ2_XXS GGUF + mmproj | no | ~112.2 GB | Tight / offload ≈37.2 tok/s ceiling | Fits ≈30.5 tok/s ceiling | No |
| GLM-5.3-Flash UD-IQ2_XXS GGUF + mmproj | yes | ~112.2 GB | Tight / offload ≈37.2 tok/s ceiling | Fits ≈30.5 tok/s ceiling | No |
| GLM-5.3-Flash Maya-M IQ2_S GGUF + vision + refusal vector | yes | ~127.0 GB | No | No | No |
| Qwen3.8 Flash-Next GSQ-RCO IQ3_S GGUF + mmproj + refusal vector | yes | ~92.8 GB | Fits ≈75.0 tok/s ceiling | Fits ≈75.0 tok/s ceiling | No |
| Qwen3.8 Flash-Next UD-IQ4_XS GGUF (4 shards) + mmproj | no | ~103.3 GB | Fits ≈67.0 tok/s ceiling | Fits ≈67.0 tok/s ceiling | No |
| Qwen3.8 Flash-Next UD-Q4_K_XL V2 GGUF (4 shards) + mmproj | yes | ~123.1 GB | No | Fits ≈45.8 tok/s ceiling | No |
| Qwen3.8-27B Mirai S 2.4-bit GGUF + mmproj + refusal control vector | yes | ~16.7 GB | Fits | Fits | Fits |
| Qwen3.8-27B Q4_K GGUF + mmproj | yes | ~22.6 GB | Fits ≈12.3 tok/s ceiling | Fits ≈12.3 tok/s ceiling | Fits ≈23.4 tok/s ceiling |
| Qwen3.8-27B UD-Q8_K_XL GGUF + mmproj | no | ~38.0 GB | Fits ≈6.7 tok/s ceiling | Fits ≈6.7 tok/s ceiling | No |
| Qwen3.8-27B BF16 GGUF + mmproj | yes | ~62.4 GB | Fits ≈3.9 tok/s ceiling | Fits ≈3.9 tok/s ceiling | No |
Token/s figures are theoretical ceilings (bandwidth ÷ bytes of active weights per token), not measurements. Real speed is usually well below. See method.
Headlines
- One M5 runs Qwen3.8 Flash-Next (125B MoE, 6B active) at IQ4_XS or alesha's abliterated IQ3_S, and GLM-5.3-Flash (320B, 18B active) at 2-bit with little headroom.
- Two M5s add the Flash-Next Q4_K_XL abliterated build and DeepSeek V4.1 Flash at IQ1_M (ISTA-DASLab build; the 110 GB Engram table stays memory-mapped on SSD).
- Not even two M5s hold the huihui DeepSeek IQ1_M build fully resident (~268 GB est.) or any original safetensors weights; those are archive copies to quantize from.
- The MacBook Pro (36 GB) runs Qwen3.8-27B at Q4 or the 2.4-bit Mirai build, plus all speech and music models.
- Video: Wan 2.2 A14B needs the card's offload flags even on an M5; HunyuanVideo 1.5 (8.3B) is the lighter video option.
Two engines matter: alesha-pro's Mirai and Maya builds need their own forked engines (mainline llama.cpp, LM Studio and Ollama will not load them), and Maya/Strata builds can't be split across two boxes with llama.cpp RPC.