Off-Grid AI has always shipped two language models and picked the right one for your hardware automatically: an 8B model ("Speed") for constrained machines and a 14B model ("Balanced") for capable ones. That covered the range from aging laptops to serious workstations.
There is now a third tier. "Expert" runs a 32B-class model, fully offline, and it is usable in a way big local models almost never are: roughly 30 tokens per second on a CPU-only machine, and well past 100 on a capable GPU. This post explains the architecture choice that made those numbers possible, and the safety engineering wrapped around it.
The Problem With Big Local AI Models
Bigger models reason better. They are also slower, and on CPU the slowdown is brutal. A traditional dense 32B model on a CPU-only machine generates maybe 2 to 4 tokens per second. That is not a premium mode. That is a spec-sheet feature nobody actually uses, because most people running local AI do not have a high-end GPU.
Shipping a big model that only works on rare hardware would violate a basic Off-Grid AI principle: capabilities have to hold up on the machines people actually own, in the conditions they actually face.
The Insight: Mixture-of-Experts Changes the Math
The Expert tier is built on a mixture-of-experts (MoE) model with roughly 35 billion total parameters but only about 3.6 billion active for any given token.
A mixture-of-experts model stores many specialized parameter groups but routes each token through only a small subset, so it needs the memory of a large model while performing computation closer to a small one.
In practice, that means:
- Memory is the real constraint, not compute. The weights need to be resident in RAM, about 22 GB of them.
- Speed behaves like a small model. The active compute per token is in the 3.6B range, which is why CPU-only machines see around 30 tokens per second instead of 2 to 4.
- Hardware gating becomes RAM-driven. 32 GB of system memory or more unlocks Expert automatically. No GPU required.
This single architectural property is the difference between a feature that exists and a feature that gets used. Expert on CPU is a first-class fast path, which is simply not true of dense models this size.
Why Not the Highest-Benchmark Model?
Some dense alternatives score higher on quality leaderboards. They were rejected because they would have been unusable on CPU, and a model you cannot run is a model that cannot help you. A "reasoning" model that emits visible thinking blocks was also rejected, because that output style conflicts with the strict answer-parsing pipeline that keeps citations verifiable. Model selection for a field system is an exercise in engineering fit, not leaderboard chasing.
The Safety Nets: How Expert Fails
How a system behaves at its best matters less than how it behaves when something goes wrong. Two guarantees shipped with the Expert tier:
1. A Free-RAM Guard
Before the model picker ever chooses Expert, it verifies there is genuinely enough free memory. If not, it quietly selects the Balanced tier instead. The device never gambles on a load it might not survive.
2. Cross-Tier Fallback
If the Expert model fails to load for any legitimate reason, the device tears down, re-picks as if Expert did not exist, loads the 14B model, and boots normally. It reports honestly that it degraded from Expert, so both you and diagnostics know exactly what happened. The device never fails to start.
The One Bright Line
There is exactly one condition where the fallback is forbidden. If the Expert model fails its integrity hash check, meaning the file is corrupt or has been tampered with, the device fails loudly and refuses to boot that model. A corrupted premium model silently masquerading as a working smaller one is precisely the kind of quiet dishonesty a trustworthy system cannot allow. Genuine failure degrades gracefully. Integrity failure stops the line.
Fail soft on resource problems. Fail loud on trust problems. Never confuse the two.
Choosing Your Tier in the Field
| Tier | Model class | Best for |
| Speed | 8B | Constrained hardware, fastest responses, quick factual lookups |
| Balanced | 14B | The default: strong quality with broad hardware compatibility |
| Expert | 35B MoE (3.6B active) | Deep, multi-step reasoning on machines with 32 GB+ RAM |
The tier selector sits in the top bar with three stops, and a tier-cap setting lets you deliberately step down when you would rather have speed than depth. Battery running low and you need quick answers, not essays? Cap it at Speed. Working through a complex electrical fault with the Master Electrician Pack loaded and time to think? Run Expert.
All three models ship on the 128 GB baseline device, and every tier, Expert included, cleared the same full quality-and-citation evaluation gate before release. A bigger model earns no exemption from the rules that make answers verifiable.
What This Means for Offline AI Buyers
When evaluating any local AI system that advertises a large model, ask three questions:
- What is the real token rate on CPU? If the honest answer is single digits, the big model is decorative.
- What happens when the model fails to load? A field device must boot regardless.
- Is model integrity verified before execution? If not, you have no idea what is actually generating your answers.
Common Misconceptions About Big Offline Models
"Bigger model, better answers, always"
A bigger model reasons more deeply, but on this platform every tier draws from the same verified corpus and passes the same citation validators. Expert improves synthesis and multi-step reasoning. It does not unlock new claims, and for quick factual lookups, Speed is often the right tool.
"You need a gaming GPU to run serious local AI"
For dense 32B models, effectively yes. For a mixture-of-experts architecture, RAM is the gate. A CPU-only machine with 32 GB of memory runs Expert at conversational speed, which is exactly why this architecture was chosen for a preparedness device: it works on hardware people actually own.
"Automatic fallback hides problems"
Fallback here is loud by design. When the device degrades from Expert to Balanced, it reports that plainly. And the one failure that never falls back silently is integrity: a model that fails its hash check refuses to run, period.
Frequently Asked Questions
What hardware unlocks the Expert tier?
32 GB of system RAM or more enables Expert automatically. A dedicated GPU with sufficient memory accelerates it dramatically but is not required for practical use.
How much storage do three models require?
The full three-tier stack occupies roughly a third of the 128 GB baseline device, with the remainder serving the corpus, source library, expansion packs, and your personal documents.
Does Expert take longer to start?
Cold-loading 22 GB of weights takes minutes on flash-drive hardware today, and drops to seconds on SSD-based devices. Once loaded, the model stays warm and responses stream immediately.
Can I force a smaller tier even on capable hardware?
Yes. The tier-cap setting lets you deliberately hold the device at Balanced or Speed when responsiveness matters more than depth, and the top-bar selector switches between unlocked tiers at will.
The Bottom Line
The Expert tier exists because one architectural insight, mixture-of-experts routing, made a 32B-class model genuinely usable on ordinary hardware, offline. It shipped only after being wrapped in the guards that define the platform: RAM-aware selection, honest degradation, loud integrity failure, and the same citation standards as every other tier.
More capability, same discipline. That is the only way capability should ever arrive on a device built for when systems fail.
This is the fourth post in a six-part series on the newest Off-Grid AI capabilities. Next: the deliberate opposite of the survival assistant, AI Playground.
