r/LocalLLaMA

[Release] AugmentedQuill 0.9.0: Open-source AI story-writing GUI

AugmentedQuill main screen in Mixed mode The open-source AI story-writing GUI AugmentedQuill has lef…

Best small coding model for completion

Hi there, I'm looking to use a small model to generate code completions simular to cursor or copilot…

Built myself a bit of a local llm workhorse. What's a good model to try out with llamacpp that will put my 56G of VRAM to good use? Any other fun suggestions?

submitted by /u/SBoots [link] [comments]…

The 4B class of 2026 (benchmark)

Bench 2 from my 18GB M3 Pro. Last week was specialists vs generalists at 7-8B (which I hosed by givi…

Qwen 3.6 27B on Strix Halo 128GB: any experiences?

I'd jump on runpod and ssh in to test my workloads, but they don't have it. Would love to know how w…

MIMO V2.5 PRO

submitted by /u/Namra_7 [link] [comments]…

For the 5 people here running vLLM on multiple R9700s, you need to patch in support for AITER Unified Attention.

I have a 4 x R9700 system on Threadripper pro, but I have never been happy with the performance of m…

Why is disabling thinking for coding models a good idea?

I've seen several people recommend disabling thinking for models when used in agent encoding, but I …

Luce DFlash: Qwen3.6-27B at up to 2x throughput on a single RTX 3090

Hey fellow Llamas, your time is precious, so I'll keep it short. We built a GGUF port of DFlash spe…

GBNF grammar tweak for faster Qwen3.6 35B-A3B and Qwen3.6 27B

Hi folks, Enjoy an optimised Qwen3.6 35B-A3B and Qwen3.6 27B for coding and general purpose - it's a…

How to run a local coding agent with Gemma 4 and Pi | Patrick Loeber

Tutorial from the Google guy, I use very similar setup (llama.cpp instead of lmstudio) submitted …

Got OpenAI's privacy filter model running on-device via ExecuTorch

Been experimenting with running OpenAI's privacy filter model on mobile through ExecuTorch. Sharing …