How to Run LLMs Locally on a Mac in 2027 (Even with 8GB or 16GB)
Last updated: November 2026. Prices, versions and benchmarks below are as of that date. Short answer: You can run an LLM locally on any Apple Silicon Mac. With 8GB of RAM, use a ~4B model like Qwen3.5 4B. With 16GB, Gem
Last updated: November 2026. Prices, versions and benchmarks below are as of that date.
Short answer: You can run an LLM locally on any Apple Silicon Mac. With 8GB of RAM, use a ~4B model like Qwen3.5 4B. With 16GB, Gemma 4 E4B (QAT) or Qwen3.5 9B run at roughly 12โ20 tokens/sec on an M2 Air. With 24GB, Mixture-of-Experts models in the 26โ35B range fit, and a base M4 Mac mini can generate around 40 tokens/sec with them. Generation speed is limited mostly by memory bandwidth, not by chip generation. The easiest tools are LM Studio and Ollama; the fastest on Apple Silicon is usually Apple's MLX.
I haven't written here in almost a year. The last thing I want to do coming back is pretend nothing changed, so let me start with the post most of you found me through.
Back in early 2025 I wrote about how to run an LLM locally on a Mac. It's still the most-read thing on this blog. It also aged badly. The Exo install command I gave you doesn't exist anymore, the models I recommended are two generations old, and the whole "you need an M4 Max" angle turned out to be the wrong question.
So this is the rewrite, and it's narrower on purpose: if you want to run LLMs locally on a Mac today, what's the least Mac you can get away with, what exactly runs on it, and how fast? At the end I'll look at 2027, and I'll try hard to keep what's announced, what's rumored and what's just me guessing in separate boxes.
Why is memory bandwidth the number that matters?
Before tools and models, one idea that will save you money.
When a local model writes text, every new token requires reading the model's active weights from memory. So generation speed is capped by roughly:
max tokens/sec โ memory bandwidth (GB/s) รท active weights read per token (GB)
That's why chip generation matters less than you'd think, and why RAM size and bandwidth matter more. Some reference points:
| Mac | Unified memory bandwidth |
|---|---|
| MacBook Neo (A18 Pro, 8GB) | 60 GB/s |
| M1 | ~68 GB/s |
| M2 / M3 | 100 GB/s |
| M4 | 120 GB/s |
| M5 | 153 GB/s |
| M6 (24GB and 32GB configs) | 170 GB/s |
| M1 Pro | 200 GB/s |
Sources: Apple's M5/MLX post for M4 and M5 (Apple ML Research), MacRumors for M6 (link), llmrun.dev for the Neo.
Look at the last row. A used 2021 M1 Pro has more bandwidth than a brand-new M5 MacBook Air. Samuel Wong made exactly this point in a thread on X: "A 2021 M1 Pro writes answers faster than a new M5 Air." If you're buying a Mac for local AI on a budget, a used Pro/Max chip with more RAM often beats a new base chip.
Quick worked example: a 4-bit 8B dense model is about 5GB. On an M4 (120 GB/s) that's a ceiling of ~24 tok/s, and real numbers land below the ceiling. Now take a Mixture-of-Experts model like Qwen3-30B-A3B: 30B total parameters, but only ~3B are active per token. At 4-bit that's under 2GB read per token, so the same chip can generate several times faster, as long as the whole model fits in RAM. That's the single biggest shift since my 2025 post, and it's why MoE shows up everywhere below.
What speed to expect on your Mac
I put the formula to work. The table below is my estimate of generation speed (4-bit, short context) for each chip, calibrated against the real measurements quoted later in this post: an M2 Air doing 12โ18 tok/s on Qwen3.5 9B and ~20 tok/s on Gemma 4 E4B, and a base M4 Mac mini doing 44 tok/s on a 35B MoE. In practice people land at about 60โ70% of the theoretical ceiling, so that's what I used. Expect ยฑ20% depending on tool, context length and what else is running.
| Chip (bandwidth) | Qwen3.5 4B | Gemma 4 E4B | Qwen3.5 9B | Gemma 4 12B | Gemma 4 26B-A4B | Qwen3-30B-A3B |
|---|---|---|---|---|---|---|
| MacBook Neo (60 GB/s) | ~13โ16 | won't fit well | won't fit | won't fit | won't fit | won't fit |
| M1 (68 GB/s) | ~15โ18 | ~11โ14 | ~7โ9 | ~5โ7 (16GB) | needs 24GB+ | needs 24GB+ |
| M2 / M3 (100 GB/s) | ~22โ27 | ~17โ21 | ~11โ14 | ~8โ10 | ~20โ23 (24GB) | ~30โ35 (24GB) |
| M4 (120 GB/s) | ~27โ32 | ~21โ25 | ~13โ16 | ~10โ12 | ~24โ28 | ~36โ44 |
| M5 (153 GB/s) | ~35โ40 | ~27โ31 | ~17โ19 | ~13โ15 | ~30โ35 | ~45โ50 |
| M6 (170 GB/s) | ~38โ44 | ~30โ34 | ~19โ21 | ~14โ16 | ~33โ38 | ~50โ56 |
| M1 Pro (200 GB/s) | ~45โ52 | ~35โ40 | ~22โ25 | ~17โ19 | ~38โ45 (32GB) | ~60โ65 (32GB) |
These are estimates, not benchmarks. If your numbers differ a lot, tell me in the comments and I'll update the table.
How much RAM do you need to run an LLM locally on a Mac?
macOS itself needs around 3.5โ4GB. Your browser and editor want more. So "fits" means the model file plus its KV cache (grows with context length) plus your apps.
Model sizes below are from the Ollama library pages for gemma4 and qwen3.5, checked Nov 2026.
8GB (MacBook Neo, older 8GB Airs)
Realistic picks: Qwen3.5 4B (3.3โ4.0GB), Qwen3.5 2B (2.7โ3.1GB), Gemma 4 E2B QAT (4.3GB).
Someone who went through this on an 8GB M1 Air summed it up well (@0xMapper_ on X): stay under about 4B parameters, use MLX 4-bit builds, and skip the default model LM Studio suggests, because it's "too heavy for 8GB". In their words, 7โ9B models load "but your mac starts swapping". They ended up on Qwen 3.5 4B, "free, offline, surprisingly decent".
The MacBook Neo is the interesting case because it's Apple's cheapest Mac (launched at $599, now $699) and has only 60 GB/s of bandwidth. I couldn't find anyone publishing measured LLM numbers on it yet. llmrun.dev estimates ~13 tok/s for Qwen3.5 4B and ~24 tok/s for Qwen3.5 2B, with a 9B model not fitting at all (source). Treat those as estimates.
And keep expectations honest for agent workflows. Sebastian Aaltonen tried a Neo for remote agentic work and wrote that it "ran out of RAM first day", with Codex, Xcode and Steam eating the memory: "8GB is not good enough for dev purposes. AI makes it even worse."
My take for 8GB: fine for a private chat assistant, summarizing, rewriting, small extraction tasks. Not for coding agents.
16GB (most MacBook Airs, base Mac minis)
This is where it gets genuinely useful. Realistic picks: Gemma 4 E4B QAT (6.1GB), Qwen3.5 9B (6.6โ7.6GB), Gemma 4 12B QAT (7.2GB) if you keep context short.
The most useful real-world story I found is Paul's (@PaulGug), who spent a week trying to run a local agent (Hermes) on a MacBook Air M2 with 16GB:
- First attempt: Qwen 3.5 9B Q4 gave about 12 tok/s with llama.cpp, and switching to an MLX server got it to 15โ18 tok/s. Still too slow for an agent: minutes between replies (post).
- What fixed it was not a faster model but less context. The stock agent started with ~30k tokens preloaded. He built a lean profile with only the tools he needed, capped context at 12k, and first turns dropped to ~3.5k tokens. With Gemma 4 E4B (QAT) he got ~20 tok/s and replies in seconds (post).
- Gemma 4 12B "just fits in RAM, but it's too slow". Qwen 2B was "a bit 'too dumb'" for agent use. E4B was his acceptable minimum.
That second bullet is the most underrated lesson in local LLMs on small Macs. Prompt processing (prefill), not generation, is what kills you on long contexts, and agents resend the whole conversation every turn. Cutting context from 30k to 12k did more than any model swap.
One more tip from a 16GB Air owner: Redouane Karzazi, running a vision model on an M4 Air, points out that Docker on a Mac runs inside a Linux VM with no access to the Metal GPU, so vLLM-in-Docker means CPU only. Go native (MLX, llama.cpp, Ollama, LM Studio).
24GB (M4/M5/M6 base with RAM upgrade, Mac mini)
This is the sweet spot going into 2027, because 24GB is enough for MoE models in the 26โ35B range.
Apple's own test: a 24GB MacBook Pro "can easily hold a 8B in BF16 precision or a 30B MoE 4-bit quantized" and stay under 18GB (Apple ML Research).
Realistic picks: Gemma 4 26B-A4B QAT (16GB), Qwen3-30B-A3B 4-bit (~17GB), and Ollama's newest addition, Thinking Machines' Inkling 2 Mini (26B total, 5.4B active, ~20GB, Apple silicon only for now, per the Ollama v0.40.17 release notes).
Real-world: @hailports runs a 35B MoE model on a base M4 Mac mini with 24GB, serving seven agents, with a median decode of 44 tok/s: "the keys are mixture-of-experts and tight memory management." In the replies they share the setup: one llama.cpp slot (-np 1), 16384 context, q8_0 KV cache, agents taking turns instead of seven parallel caches.
That's a $799-class machine doing what needed a workstation two years ago.
Which tool should you use: Ollama, LM Studio, MLX, llama.cpp or Exo?
There's no single best tool. Here's how I'd choose in late 2026.
| If you want... | Use |
|---|---|
| A GUI, browse and try models, zero terminal | LM Studio |
| A local API for apps/agents, simplest CLI | Ollama |
| Maximum speed on Apple Silicon, Python | mlx-lm |
| Maximum control, GGUF, tight memory tuning | llama.cpp |
| Combining several Macs into one | Exo (or LM Studio's distributed mode, coming) |
Ollama (now on MLX)
The big change this year: since v0.19 (March 2026) Ollama on Apple Silicon runs on Apple's MLX framework. Ollama's own benchmark on Qwen3.5-35B-A3B went from 58 to 112 tok/s decode and 1,154 to 1,810 tok/s prefill (Ollama blog). Alan West reproduced almost exactly that on an M4 Max 64GB: 57.8 โ 111.4 tok/s (dev.to).
Two caveats people miss:
-
The engine depends on the model format, not the app version. MLX tags (e.g.
gemma4:e4b-mlx,gemma4:12b-nvfp4) run on MLX; GGUF tags (q4_K_M,qat) still run on the llama.cpp engine. A model you pulled in 2025 does not get faster just because you updated (Level Up Coding). -
On small Macs the MLX tags can be bigger. Look at Gemma 4 E4B: the
e4b-it-qatGGUF is 6.1GB, thee4b-mlxtag is 9.5GB. On a 16GB Air, the smaller QAT file can be the better choice even if MLX is faster per token. The original MLX preview asked for more than 32GB of unified memory.
# 8GB
ollama run qwen3.5:4b
# 16GB
ollama run gemma4:e4b-it-qat
ollama run qwen3.5:9b
# 24GB
ollama run gemma4:26b-a4b-it-qat
# Hook a local model into a coding agent
ollama launch claude --model qwen3.5
ollama run <model> --verbose prints the eval rate (tok/s) after each answer, which is how you should measure.
LM Studio
Still the friendliest entry point. Two things worth knowing in 2026:
- It ships a CLI,
lms, once you've opened the app at least once (docs):
lms get # search and download
lms ls # what's on disk
lms load <model> --context-length=8192
lms server start # OpenAI-compatible local server
Setting --context-length explicitly is the single best thing you can do on a 16GB Mac (see Paul's story above).
- LM Link lets you run a model on one machine and use it from another (laptop, iPhone) over an end-to-end encrypted network built with Tailscale (docs). This matters for the 2027 section below.
mlx-lm
Apple's own library, and usually the fastest single-machine option on Apple Silicon. One independent comparison on an M4 Max 128GB (cited by Level Up Coding) had the same model at 48.1 tok/s in pre-MLX Ollama, 72.4 in raw llama.cpp and 131.8 in mlx-lm.
pip install mlx-lm
# chat in the terminal
mlx_lm.chat --model Qwen/Qwen3-8B-MLX-4bit
# one-off generation (prints tokens/sec at the end)
mlx_lm.generate --model mlx-community/Llama-3.2-3B-Instruct-4bit --prompt "Explain KV cache in 3 sentences"
# OpenAI-compatible server
mlx_lm.server --model Qwen/Qwen3-8B-MLX-4bit
# quantize a Hugging Face model to 4-bit yourself
mlx_lm.convert --model mistralai/Mistral-7B-Instruct-v0.3 -q
(Commands from the mlx-lm README.)
If you have an M5 or newer, MLX uses the GPU Neural Accelerators. Apple measured, on 24GB MacBook Pros, M5 vs M4 (source):
| Model | Time to first token | Generation speed | Memory |
|---|---|---|---|
| Qwen3-8B 4-bit | 3.97ร faster | 1.24ร faster | 5.6GB |
| Qwen3-14B 4-bit | 4.06ร faster | 1.19ร faster | 9.2GB |
| gpt-oss-20b MXFP4 | 3.33ร faster | 1.24ร faster | 12.1GB |
| Qwen3-30B-A3B 4-bit | 3.52ร faster | 1.25ร faster | 17.3GB |
Read that carefully: the Neural Accelerators made prefill ~4ร faster, but generation only ~1.2ร faster, which matches the bandwidth jump (120 โ 153 GB/s). Same lesson again: generation is a memory-bandwidth problem. Neural Accelerators need macOS 26.2 or later.
llama.cpp
Hit v1.0 this year and has a new installer and a simpler llama command (README):
curl -LsSf https://llama.app/install.sh | sh
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF # chat
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF # OpenAI-compatible server
-hf pulls GGUF files straight from Hugging Face, so for Gemma 4 E4B QAT you'd use -hf unsloth/gemma-4-E4B-it-qat-GGUF:UD-Q4_K_XL (Unsloth recommends --temp 1.0 --top-p 0.95 --top-k 64 for Gemma 4, docs).
Use llama.cpp when you need control: context size, KV cache quantization (q8_0 halves KV memory, like hailports' setup), speculative decoding. Kyle Howells measured Gemma 4 26B-A4B on an M1 Max going from 58.2 to 69.2 tok/s with multi-token prediction enabled (via Kunal Ganglani's write-up).
Exo (correction to my 2025 post)
In my old post I gave you curl ... install.exo.sh and an exo run ... --devices command. Those don't work. Sorry. Today Exo is a macOS menu-bar app (README):
brew install --cask exo
# or download EXO-latest.dmg from the README
Open it on each Mac on the same network. They discover each other automatically, and the dashboard plus an OpenAI/Claude/Ollama-compatible API run at http://localhost:52415. Requirements: Apple silicon, macOS Sequoia 15.1+, and every Mac must run the same Exo version or they won't join.
Be realistic about what Exo is for. Over Wi-Fi it lets you fit a model that doesn't fit on one machine; it won't make a small model faster. The headline numbers (Exo claims up to 1.8ร on 2 devices and 3.2ร on 4 with tensor parallelism) come from Thunderbolt 5 + RDMA, which needs macOS 26.2+ and TB5 Macs (M4 Pro and up). Two 16GB Airs over Wi-Fi are a fun experiment, not a speed upgrade.
How do you measure tokens per second on your Mac?
Don't trust anyone's tok/s, including my table above. Same prompt, same model, your Mac:
# Ollama
ollama run gemma4:e4b-it-qat --verbose "Explain how a hash map works, in 150 words."
# mlx-lm
mlx_lm.generate --model Qwen/Qwen3-8B-MLX-4bit \
--prompt "Explain how a hash map works, in 150 words." --max-tokens 256
Look at two numbers: prompt eval rate (prefill) and eval rate (generation). Then open Activity Monitor โ Memory and check the Memory Pressure graph. Yellow is okay. Red, and you're swapping.
If your generation number is far below the estimate table, the usual suspects are: a long context, a GGUF tag where an MLX tag exists, or memory pressure from other apps.
Common errors when running local LLMs on a Mac (and fixes)
model requires more system memory (X GiB) than is available (Y GiB) (Ollama)
Pick a smaller tag (qat or a smaller size), or quit memory-hungry apps. On 16GB, a 12B model plus Chrome plus an IDE is a common way to hit this.
[WARNING] Generating with a model that requires ... (mlx-lm)
The model is large relative to RAM and will be slow. If it does fit, raising the GPU wired memory limit can help (macOS 15+):
sudo sysctl iogpu.wired_limit_mb=N # N > model size in MB, < total RAM
This resets on reboot. Leave real headroom for macOS.
"I updated Ollama and nothing got faster."
You're probably on a GGUF tag. Check which engine actually ran:
grep -oE "llama-server|mlxrunner" ~/.ollama/logs/server.log | sort | uniq -c
(from Level Up Coding). Pull an -mlx or -nvfp4 tag if you have the RAM.
Exo dashboard shows DEVICE ON ANOTHER EXO VERSION
Update every Mac to the same version. Also check System Settings โ Privacy & Security โ Local Network is on for EXO. Guest Wi-Fi often blocks device discovery; try Ethernet or a Thunderbolt cable.
Agent works for two turns, then crawls
Context. Cap it (lms load --context-length, -c in llama.cpp, --max-kv-size in mlx-lm) and slim down the system prompt and tool list.
"Is swap killing my SSD?"
Not by itself. As Chhinsoportrakhou puts it on Towards AI, "using swap memory does not automatically destroy your Mac's SSD". The real problem is constant memory pressure making everything slow. Pick a model that fits.
What will change for local LLMs on Macs in 2027?
I'll split this into three buckets, because most "future of AI" posts blur them.
Already announced (fact)
- Prefill got fixed. M5's Neural Accelerators made time-to-first-token ~4ร faster than M4 in Apple's tests. M6 (announced Oct 2026, in the 14" MacBook Pro) goes to 170 GB/s on its 24/32GB configs (MacRumors). Long-context and agent workloads were the weak spot of Macs; that gap is closing fast.
- Models got smaller for the same quality. Gemma 4 ships official quantization-aware trained (QAT) versions: the 26B MoE runs in ~15GB, E4B in ~5GB, E2B text-only under 1GB on mobile formats (Google, Unsloth). Ollama reports NVIDIA's NVFP4 format roughly halves the quality loss of 4-bit vs q4_K_M on Gemma 4 12B, while generating ~20% faster (Ollama).
- Apple is openly courting local inference. At WWDC26, Apple had a session on distributed inference with MLX: RDMA over Thunderbolt 5, a 1T-parameter Kimi K2.6 running across four Mac Studios, and Qwen 3.6 27B running nearly 3ร faster on four machines than one (WWDC26 session 233). LM Studio ran the same demo and accessed it "from a MacBook Neo and iPhone using LM Link".
Rumored (credible reporting, not confirmed)
- M6 Pro/Max may skip to M7 in 2027. Bloomberg reporting (via MacRumors) said M6 Pro and Max weren't planned and the next high-end chips would be M7 in 2027, on TSMC's 2nm-class process with new packaging. The same report predicted ~200 GB/s for base M6; it shipped at 170, so take bandwidth rumors with salt.
- A MacBook Neo with A19 Pro and 12GB is reportedly being tested (Gurman, via TechRepublic). 12GB vs 8GB is the difference between "4B max" and "comfortable E4B/9B with room for a browser".
- Counter-force: memory is getting more expensive. The Neo went from $599 to $699 in June 2026, reportedly due to memory costs. Cheap RAM is not guaranteed.
My predictions (speculation, label it as such)
- "Small Mac" will mean 16GB, and 16GB will mean today's 24GB class. Not because Macs get much more RAM, but because QAT + MoE keep shrinking the memory per unit of quality. A year ago a useful agent model needed 24GB+. Today Paul runs one on a 16GB M2 Air. I expect a Gemma-4-26B-quality model to fit comfortably in 16GB by late 2027. Confidence: medium.
- Generation speed on base chips will improve slowly; prefill fast. Bandwidth is rising ~10โ25% per generation (120 โ 153 โ 170). MoE with fewer active parameters will do more for tok/s than the chip will. Confidence: high.
- The home Mac becomes the inference server, the laptop becomes the client. Between LM Link, Exo, MLX distributed and Ollama's API, the pattern "Mac mini/Studio in a closet, thin MacBook (even a Neo) everywhere" is already working. I think in 2027 that's the default setup for people who care about privacy or API bills, the way a home NAS became normal for photos. Confidence: medium.
- Will inference "move to Macs" from data centers? For individuals and small teams, partially, yes, especially for agents running all day where API costs add up. For serving many users at once, no: Macs win on memory per dollar and per watt, not on batch throughput. Confidence: medium-high.
TL;DR
- 8GB: Qwen3.5 4B or Gemma 4 E2B. Chat and text tasks, not agents.
- 16GB: Gemma 4 E4B QAT or Qwen3.5 9B. Agents work if you keep context small.
- 24GB: 26โ35B MoE models (Gemma 4 26B-A4B, Qwen3-30B-A3B). This is the 2027 sweet spot.
- Buying? Bandwidth and RAM beat chip generation. A used M1/M2 Pro with 32GB can beat a new base Air.
- Tools: LM Studio to start, Ollama for apps, mlx-lm for speed, llama.cpp for control, Exo when one Mac isn't enough.
FAQ
Can an 8GB Mac run an LLM?
Yes. Models up to about 4B parameters at 4-bit (Qwen3.5 4B, Gemma 4 E2B) fit in 8GB. 7โ9B models load but cause swapping. Fine for chat and text tasks, not for coding agents.
What is the best local LLM for a 16GB MacBook Air?
As of November 2026, Gemma 4 E4B (QAT, ~6GB) and Qwen3.5 9B (~7GB). One M2 Air user measured ~20 tok/s with Gemma 4 E4B QAT and 12โ18 tok/s with Qwen3.5 9B.
Is MLX faster than Ollama on a Mac?
Ollama has used MLX on Apple Silicon since v0.19 (March 2026), but only for MLX-format model tags. GGUF tags still use the llama.cpp engine. Raw mlx-lm is usually the fastest option on a single Mac.
Does the M5 chip make local LLMs faster?
Mostly for prompt processing. Apple measured up to ~4ร faster time-to-first-token vs M4, but only ~1.2ร faster generation, because memory bandwidth rose from 120 to 153 GB/s.
Can I combine two Macs to run a bigger model?
Yes, with Exo or MLX distributed. Over Wi-Fi it lets a bigger model fit; real speedups need Thunderbolt 5 with RDMA (macOS 26.2+).
If you run any of these on your own Mac, drop your model, RAM and tok/s in the comments. I'll add the best ones to the table.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes โ full credit and traffic to the original publisher.