Running Flux locally on a Mac: install, commands and two models compared

People write about local image generation in one of two ways. Either "it installs in three clicks, why pay for anything," or "it doesn't work on a laptop." Both are equally useless when you're sitting in front of an empty terminal and just want a picture out of it.

So here's the manual instead. Every command below was actually run on this machine, every number comes from the log of a specific run: Apple M5, 16 GB of unified memory, macOS 26.5, stable-diffusion.cpp build master-650.

We'll install two models from the same family, and the difference between them is the whole point. Flux.1-schnell draws a frame from scratch out of text. Flux.1-Kontext-dev takes a finished picture and changes exactly what you asked for, leaving the rest alone. Different jobs — one doesn't replace the other.

Why keep a model on your own machine

An honest list of exactly three reasons, because there is no fourth one.

• Free. There is no spend counter at all. Trying thirty variations of a scene costs you time and nothing else. That changes behaviour: with a cloud API you think before you press enter, locally you just run batches in the background.
• Offline. No internet needed at any point once the weights are downloaded. Planes, cabins, dead VPNs, corporate networks that block everything — the model doesn't care.
• Private. The picture never leaves your machine. This is the one argument the cloud cannot beat: NDA material, someone else's mockups, internal screenshots.

And now the part these articles usually skip. Local will not be faster than the cloud. In an earlier benchmark on identical prompts, local Flux produced a frame in 172 seconds and GPT Image in 167 — and the cloud model was visibly better at composite scenes. Local wins on cost, autonomy and privacy. Not on speed, and not on quality.

Step 1. Build the engine

xcode-select --install
brew install cmake

git clone --recursive https://github.com/leejet/stable-diffusion.cpp ~/Tools/stable-diffusion.cpp
cd ~/Tools/stable-diffusion.cpp
mkdir build && cd build
cmake .. -DSD_METAL=ON
cmake --build . --config Release -j

Two places where people trip.

--recursive is mandatory. The ggml compute core is a submodule; without it the build dies on missing headers. Already cloned without it? git submodule update --init --recursive.

-DSD_METAL=ON is what turns the GPU on. The .cpp suffix in the project name is misleading — it reads like "the CPU version, therefore slow." It isn't. The first line of the log says:

ggml_metal_device_init: GPU name: MTL0 (Apple M5)

The GPU does the work. The CPU gets a separate and fairly humiliating job: cleaning up after what's broken in Metal. More on that below, and it's the most important section here.

Step 2. Download the weights

Flux is not one file. It's four, and all four are required.

Two encoders is a Flux architecture thing: CLIP gives the general mood of the prompt, T5 reads the wording word by word. That's also where its strength comes from — you write long descriptive English sentences, not comma-separated tags.

mkdir -p ~/Tools/sd-models/flux && cd ~/Tools/sd-models/flux

curl -L -C - -o flux1-schnell-Q4_K_S.gguf \
  https://huggingface.co/city96/FLUX.1-schnell-gguf/resolve/main/flux1-schnell-Q4_K_S.gguf

curl -L -C - -o t5xxl-Q4_K_M.gguf \
  https://huggingface.co/city96/t5-v1_1-xxl-encoder-gguf/resolve/main/t5-v1_1-xxl-encoder-Q4_K_M.gguf

curl -L -C - -o clip_l.safetensors \
  https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/clip_l.safetensors

curl -L -C - -o ae.safetensors \
  https://huggingface.co/second-state/FLUX.1-schnell-GGUF/resolve/main/ae.safetensors

About the VAE: the obvious place to get it would be the official Black Forest Labs repo, but it's license-gated and returns 401 without a token and an accepted agreement. Hence the second-state mirror — same file, byte for byte.

The gotcha that costs you an evening

curl can break in the middle of a large file and exit with code 0. The file looks complete, the GGUF header is there, and the model simply won't load. The error you get is unhelpful:

read tensor data failed

Check the size against the header the server sends:

URL=https://huggingface.co/city96/FLUX.1-schnell-gguf/resolve/main/flux1-schnell-Q4_K_S.gguf
EXPECTED=$(curl -sIL "$URL" | awk 'BEGIN{IGNORECASE=1} /^content-length:/{l=$2} END{print l+0}')
ACTUAL=$(stat -f%z flux1-schnell-Q4_K_S.gguf)
[ "$ACTUAL" -eq "$EXPECTED" ] && echo "OK" || echo "TRUNCATED: $ACTUAL of $EXPECTED"

-C - resumes from where it broke, so re-running loses nothing. If your connection drops regularly, add --speed-limit 51200 --speed-time 30: that pair kills a dead connection after half a minute instead of hanging on a 30-minute timeout.

One more thing that isn't about the tooling: don't generate while the weights are downloading. The model pushes memory into swap, and under swap the network on this machine dies — HuggingFace starts serving kilobytes per second, which then gets blamed on "bad internet."

Step 3. The first picture

~/Tools/stable-diffusion.cpp/build/bin/sd-cli -M img_gen \
  --diffusion-model ~/Tools/sd-models/flux/flux1-schnell-Q4_K_S.gguf \
  --vae        ~/Tools/sd-models/flux/ae.safetensors \
  --clip_l     ~/Tools/sd-models/flux/clip_l.safetensors \
  --t5xxl      ~/Tools/sd-models/flux/t5xxl-Q4_K_M.gguf \
  -p "A vintage green enamel mug standing on a weathered wooden windowsill, warm morning light, a rainy blurred street outside the window, photorealistic, 50mm lens, shallow depth of field, no legible text anywhere" \
  --cfg-scale 1.0 --sampling-method euler --steps 4 \
  -W 1024 -H 1024 --seed 42 \
  --vae-on-cpu --diffusion-fa \
  -o ~/Desktop/first.png

Two flags you must not touch.

--cfg-scale stays at 1.0

The 7.0 that floats around every Stable Diffusion tutorial turns the picture into coloured noise here. Schnell is a distilled model: it was trained to hit the target in a handful of steps without a guidance mechanism. At 1.0 that mechanism is effectively off, which is the correct mode.

Same root cause for the second surprise: negative prompts do nothing. Not "work poorly" — they physically don't participate in the computation. "No people in frame" has to become "an empty street."

--vae-on-cpu is the most expensive thing I know about this stack

Without it you get a blank white frame. Not an error, not a crash — a white rectangle, and the log cheerfully says:

save result image (success)

Flux's VAE decoder returns NaN on Metal and the result silently collapses. You can spot it instantly by file size: a broken PNG is about 17 KB, a live one is 450 KB and up (mine came out at 1.7–1.9 MB). If you see seventeen kilobytes, don't rewrite the prompt and don't re-download the weights — just add the flag.

The flag moves decoding to the CPU. That isn't free: on a 1024×1024 frame the CPU VAE costs about 58 seconds, and those 58 seconds are in every single picture no matter how many steps you run.

How many steps you actually need

The common "put it on twenty steps for more detail" advice is not just useless for schnell, it's actively wrong. Three runs, same prompt, same seed, only the step count differs:

Three things fall out of this table, none of them obvious up front.

A step costs exactly 26.3 seconds, linearly. No warm-up: the first step costs the same as the eighth. So you can do the arithmetic in your head — 58 seconds plus 26 per step.

At one step, two thirds of the time isn't drawing. 57 seconds out of 84 is the CPU VAE, i.e. working around the Metal bug. On fast modes you're mostly paying to patch Metal, not to run the model.

Eight steps don't give you "the same, but better." They give you a different frame: different angle, different light, different street outside the window. For a distilled model the step count is another dimension of the scene, like the seed. So "let me add steps to fix that weird handle" doesn't work — you'll get a different mug.

Practical default: 4 steps for real work, 1 step for browsing ideas. One step in a minute and a half already gives a usable frame; run a dozen concepts that way, then re-run the good one at 4 steps with the same seed.

One more thing from the 8-step frame: the model decorated the buildings with shop signs whose letters are not letters. Flux renders plausible gibberish instead of text, and the more steps, the more eagerly. Put no legible text anywhere in the prompt — and if you genuinely need readable text in the image, a local model is the wrong tool.

The second model: editing a finished frame

Here's the reason to keep a second model on disk at all. A generator has a built-in limitation you can't prompt your way around.

Picture this: the frame came out well, you like everything, but you'd like the mug replaced with a cactus. Asking the generator for that is pointless — it will draw a new picture: different light, different window, different street. You can burn an hour on retries and never get the same scene back, because "the same scene" doesn't exist in its world.

Flux.1-Kontext-dev works differently: it takes a finished image plus an instruction about what to change. Same command as before with three differences — a different model file, a -r flag pointing at the source image, and a prompt in the imperative:

~/Tools/stable-diffusion.cpp/build/bin/sd-cli -M img_gen \
  --diffusion-model ~/Tools/sd-models/flux-kontext/flux1-kontext-dev-Q4_K_S.gguf \
  --vae        ~/Tools/sd-models/flux/ae.safetensors \
  --clip_l     ~/Tools/sd-models/flux/clip_l.safetensors \
  --t5xxl      ~/Tools/sd-models/flux/t5xxl-Q4_K_M.gguf \
  -r first.png \
  -p "Replace the green enamel mug with a small potted cactus in a terracotta pot, keep everything else exactly the same" \
  --cfg-scale 1.0 --guidance 2.5 --sampling-method euler --steps 8 \
  -W 1024 -H 1024 --seed 42 \
  --vae-on-cpu --clip-on-cpu --diffusion-fa \
  -o cactus.png

Kontext weights live separately, the encoders and VAE are reused — it's exactly one more file:

mkdir -p ~/Tools/sd-models/flux-kontext && cd ~/Tools/sd-models/flux-kontext

curl -L -C - -o flux1-kontext-dev-Q4_K_S.gguf \
  https://huggingface.co/QuantStack/FLUX.1-Kontext-dev-GGUF/resolve/main/flux1-kontext-dev-Q4_K_S.gguf

Same gated-repo trap here: the city96 Kontext build needs an accepted license and returns 401 without a token. The QuantStack mirror is open.

The mug became a cactus, and the street behind the glass, the frame, the raindrops, the windowsill planks and the light all stayed exactly as they were. None of those were mentioned in the prompt — the model understood it wasn't asked to touch them.

Kontext is not distilled — unlike schnell it's an ordinary model, so it wants both --guidance and more steps. Eight is the working minimum; complex edits are worth twenty.

--clip-on-cpu pushes the text encoders into regular RAM. Kontext's diffusion part is heavier, and on 16 GB that flag decides whether everything fits at once: it splits the load into 6.5 GB on the GPU and 3.5 GB in RAM instead of nearly ten in one place.

It preserves the scene, not the pixels

It looks like the model neatly cut out the mug and pasted a cactus. In reality the whole frame is re-rendered, just with the original in view. I measured the difference pixel by pixel:

The "untouched" background is in fact touched — you just can't see it. Practical consequence: you cannot layer a Kontext result over the original, and you cannot use it to edit a photo where pixel authenticity matters. For an article cover it's irrelevant. For a document or someone else's photo it isn't.

What it costs in time

Editing is more expensive than generating, for an obvious reason: the model has to read the source image first, not just the text. One run, 1024×1024, eight steps:

Nine and a half minutes per edit is not "playing around." It's a go-make-coffee workflow, and you should plan around that.

The result is reproducible, though. I ran the same request twice: the first time the machine was thrashing and each step took two minutes, the second time memory was free. Both runs produced a byte-identical PNG. Machine load affects the time, not the picture.

Q4 vs Q6: are three gigabytes worth it

The file name tells you how hard the model was squeezed. Quantization stores weights at reduced precision: Q4 gives each number roughly four bits instead of sixteen, Q6 roughly six. Higher number, closer to the original, fatter file.

For Kontext the gap is three gigabytes, and on a 16 GB machine that isn't an abstraction. I ran both on the same picture with the same prompt and seed.

You can't tell the outputs apart by eye, and that's measurable, not a feeling:

• SSIM — 0.983 out of 1.0
• PSNR — 33.9 dB
• 1% of pixels differ by more than 16 levels; mean difference across the frame is 1.7 levels out of 255

For comparison, the edit itself changed a third of the frame. The difference between quants is thirty times smaller than the difference you asked for.

Speed, interestingly, is a wash too: 58.7 s/step for Q4 against 57.0 for Q6 is run-to-run noise, not an advantage. The extra three gigabytes don't slow the math down. They eat memory — and that's the real price.

So on 16 GB the answer is simple: take Q4. Three gigabytes of disk is the lesser problem; the worse one is that those same three gigabytes move the machine closer to swap. And swap on these models costs you multiples, not percentages: I left a second generation process running by accident, and a 58-second step turned into 142.

Cheat sheet

Two shell functions save you from retyping the long command:

FLUX=~/Tools/sd-models/flux
SD=~/Tools/stable-diffusion.cpp/build/bin/sd-cli

# generate from scratch: img "prompt" [steps]
img() {
  $SD -M img_gen \
    --diffusion-model $FLUX/flux1-schnell-Q4_K_S.gguf \
    --vae $FLUX/ae.safetensors --clip_l $FLUX/clip_l.safetensors \
    --t5xxl $FLUX/t5xxl-Q4_K_M.gguf \
    -p "$1" --cfg-scale 1.0 --sampling-method euler --steps "${2:-4}" \
    -W 1024 -H 1024 --vae-on-cpu --diffusion-fa \
    -o ~/Desktop/img_$(date +%H%M%S).png
}

# edit a finished frame: edit source.png "what to change"
edit() {
  $SD -M img_gen \
    --diffusion-model ~/Tools/sd-models/flux-kontext/flux1-kontext-dev-Q4_K_S.gguf \
    --vae $FLUX/ae.safetensors --clip_l $FLUX/clip_l.safetensors \
    --t5xxl $FLUX/t5xxl-Q4_K_M.gguf \
    -r "$1" -p "$2" --cfg-scale 1.0 --guidance 2.5 \
    --sampling-method euler --steps 8 -W 1024 -H 1024 \
    --vae-on-cpu --clip-on-cpu --diffusion-fa \
    -o ~/Desktop/edit_$(date +%H%M%S).png
}

And one habit that saves hours — check swap before starting a heavy model:

sysctl -n vm.swapusage

If it's full, don't hunt for speed by tuning steps and resolution. Let memory free up first. On 16 GB of unified memory the rule is blunt but accurate: either the model, or everything else.

What I actually use it for

Local Flux isn't a replacement for the cloud, it's a separate tool with its own niche. It's not faster and not better — it's free, autonomous, and it shows your pictures to nobody.

• Drafts. Checking whether a cover concept reads at all — ten variants at one step each while the tea brews. The one that works goes to a cloud model for the final.
• Edits. "Keep everything, just change this" requests go to Kontext, because a generator answers those with a new picture.
• Anything that can't leave the machine. No alternative here.

Fifteen minutes to build, an evening to download, 17 GB on disk — and after that it just works, with no counter and no internet.

Originally published at klukyanov.ru.

Shorter weekly write-ups (in Russian) — on Telegram.