r/LocalLLaMA

I can’t believe I can say “ugh I don’t feel like fixing this function, it’s too complex” and I can literally just tell my computer to fix it for me. I didn’t understand what they meant by “people will start paying for intelligence” but now I do.

And in this case it’s free! Aside from the electricity haha I hope these things aren’t conscious. I…

Structured CoT: Shorter Reasoning with a Grammar File

submitted by /u/Thrumpwart [link] [comments]…

Qwen3.6-35B-A3B KLDs - INTs and NVFPs

https://preview.redd.it/c76w57d1yexg1.png?width=1482&format=png&auto=webp&s=1164d8bc3e2e8a4157f26dd5…

Higher precision or higher parameter count

I’m wondering if we take models of the same family (e.g qwen3.5 moes). And we compared ggufs that ar…

Quant Qwen3.6-27B on 16GB VRAM with 100k context length

https://preview.redd.it/tblmrwxkbexg1.png?width=1193&format=png&auto=webp&s=6dea1e6684e75e22852d57c0…

2x RTX 6000 build during an extended bench test

CPU at 100% and both GPUs at 535w cap. Recorded for noise and thermals documentation. This is as loa…

Qwen 3.6 35b a3b Q4 vs qwen 3.6 27b q6, on m5 pro 64gb

Tried to test the two versions of models in my own m5 pro 64, curated the results on claude, not an …

FINAL-Bench/Darwin-36B-Opus · Hugging Face

https://huggingface.co/bartowski/FINAL-Bench_Darwin-36B-Opus-GGUF Darwin-36B-Opus is a 36-billion-pa…

Qwen3.6 35b a3b Particle System

Started testing Qwen3.6 35b a3b. I let it code a particle System with my Pi Agent. It just made one …

GLM 5.1 Locally: 40tps, 2000+ pp/s

After some sglang patching and countless experiments, managed to get reap-ed nvfp4 version running s…

FP4 inference in llama.cpp (NVFP4) and ik_llama.cpp (MXFP4) landed - Finally

Both llama.cpp and ik_llama.cpp now have FP4 support — but with different flavors worth knowing abou…

PSA: Ubuntu 26.04 makes it easier to get started with AMD XDNA2 NPU

https://lemonade-server.ai/flm_npu_linux.html submitted by /u/jfowers_amd [link] [comments…