In development
Status: in development. The engine work described here is on a development branch and is not in any released or TestFlight build of Local Mind. No release date is promised, and no speed is claimed — see what is not claimed.
DeepSeek V4.1-Flash is a 748B-parameter mixture-of-experts model with 16B active per token. It is far larger than any Mac's memory, so it is read from storage as it runs rather than held resident, and that makes the size of the weight file the thing that matters most.
The quantisations published for this model range from about 217 to 474 GiB. Smaller is not merely cheaper to store: on a machine that streams the weights, fewer bytes per token is the dominant term.
| quantisation | size | shards |
|---|---|---|
| UD-IQ1_S | 217.0 GiB | 5 |
| UD-IQ1_M | 233.2 GiB | 5 |
| UD-IQ2_XXS | 279.8 GiB | 6 |
| UD-Q2_K_XL | 311.8 GiB | 7 |
| UD-IQ3_XXS | 367.6 GiB | 7 |
| UD-IQ3_S | 394.5 GiB | 8 |
| UD-Q3_K_XL | 451.8 GiB | 8 |
| UD-Q4_K_XL | 466.3 GiB | 9 |
| UD-Q8_K_XL | 473.5 GiB | 9 |
This model carries an n-gram memory — a very large lookup table consulted per token, separate from the transformer weights. In these exports it is a single tensor of 768,022,850 rows that lives alone in the last shard, while the constants describing how to index it live in the first shard. A loader therefore has to carry a description read from one file to a table held in another.
Local Mind already decoded this model from its own weight export. What was missing was that the community exports spell the same indexing constants under different names. They are now read under both spellings, so a file written by either converter loads.
One detail worth recording, because it is the kind of thing that passes review and then fails quietly: the row count of each half of the table could be computed two ways — from the stored offsets, or by summing the bucket sizes. The loader checks those two against each other, so it deliberately reads one of them rather than deriving it from the other. A check whose two sides come from the same number cannot fail.
Everything else on this page is about Local Mind's own engine. Separately, the six-shard GSQ-RCO Coder export of this model runs under stock llama.cpp on two machines here, and those rates are published in the machines table rather than claimed here: 24.8 tok/s decode with 207 t/s prefill on an M1 Ultra 128 GB (Metal, four slots, 256K context), and 18.5 tok/s with 183 t/s on a Ryzen AI Max+ 395 128 GB (ROCm/HIP, four slots, 1M context).
The two hosts reached different backends for a reason worth recording. On Vulkan the AMD build cannot serve this model at 1M at all: it aborts in ggml-backend with pre-allocated tensor (cache_k_l20 (reshaped) (view)) ... cannot run the operation (SET_ROWS), which is a backend op-coverage limit rather than memory — the attempt ran with 119 GB free. The same box built for ROCm/HIP loads 1M without complaint, at a measured prefill −21% and decode −9% against its own Vulkan numbers at an identical 256K (235 t/s and 20.2 tok/s). At 1M ROCm is no slower than at 256k, because KV is cheap here: one KV head, 12.9 GB at 1M against 3.3 GB at 256K. So the 4x window costs a round-off once the backend tax is paid.
Prefill still belongs with its prompt length, and so does the ubatch: the Mac ran -ub 1024 until 4 November only because the AMD host had been tuned to it, and interleaved measurement puts that 5.7% below the default at 1,005 tokens (183.6 against 194.1) while the two agree at 6,405.
That export is 182.7 GB — 72.1 GB of transformer weights plus a 110.6 GB n-gram table — and the table is memory-mapped and faulted a row at a time, which is why a 182.7 GB file decodes on a 128 GB machine at all. Two figures above need a note beside them. The 748B parameter count is the publisher's; summing that export's own tensor table gives 476.7B — 271.8B routed experts, 197.3B in the n-gram table, 7.6B elsewhere — and about 16B of it is read per decoded token, 8.5B of those routed experts. The machines table publishes the summed figure. And the UD-IQ2_XXS 279.8 GiB row in the quantisation table is a different export from this file: the Coder pack is 170.2 GiB in the same IQ2_XXS type. At 2.06 bits per weight the transformer stack is 72 GB in either pack, so the ~110 GiB between them is the table: it works out to roughly 9 bits per entry for the UD pack against 4.5 here, which is a difference in the table's precision, not in the model. Both are recorded rather than reconciled.
No decode or prefill speed, and this one is worth being specific about rather than merely cautious. The instrument was built to check correctness, not to time anything, and what it times is mostly the operating system's file cache. Running the identical walk a second time, with no change to the code, made the first position roughly eight times faster and halved the next three, purely because the first run had left the data cached. The per-position byte count is constant, yet its apparent read rate moves more than eightfold on cache state alone. A number from this path would describe how warm the machine happened to be, so none is published — and the faster number is the one to distrust most.
Not parity. Agreement at 8 of 12 positions is cross-quantisation agreement with a differently-built export, and the test deliberately asserts no threshold: the correct agreement rate between two different quantisations is not known in advance, so any threshold would be a number chosen to pass. The four positions that differ are spread through the sequence rather than clustered at the start. An earlier four-position run of the same walk agreed at 3 of 4 and differed only at the first position, which read like a story about missing context; twelve positions show that was an artefact of four samples, and the figure here is quoted with its sample size for that reason.
Not an offering. None of these weight files is in Local Mind's catalogue, and nothing here says the app will download or serve them.