r/LocalLLaMA

FYI llamacpp server can hot swap models now-a-days in under 30sec

See this question at least a handful of times when browsing new and in the comments, llamacpp has on…

Nemotron 3 Ultra is available on HuggingChat

impressive speed/performance ratio! served by togetherAI :) submitted by /u/paf1138 [link] …

Qwen 3.6-27B on vLLM with dual RTX 3090s: looking for launch parameters

Hi everyone. Please share your working launch commands for running Qwen 3.6-27B via vLLM on dual RTX…

I implemented KVarN in my llama.cpp fork and ran KLD benchmarks. It's promising!

Saw this post here yesterday: KVarN: new KV-cache quant from Huawei. 3–5× KV cache compression with …

I just realized how good MoE models are for consumer hardware

I've been tinkering around with LLM for a while now, started with LM Studio like probably all of us …

Suggestion - this sub should have post flairs that mention the amount of vram/unified ram

The amount of fast ram is the single most important factor for llm use. There are lots of people tha…

Intel B70 vs AMD R9700: Has anyone actually tested the noise levels (dB) at full load?

Both 32GB GDDR6. Intel somewhat slower but lower TDP (230W) and a little cheaper. I wish AMD did off…

438 USD for a 3080 20GB isn’t bad

submitted by /u/xw1y [link] [comments]…

Microsoft should've released something like Qwen3.6-27B / Gemma-4-31B already. They released MAI models now

Did they abandon Phi series? I remember that few were expecting for Phi-5. I see that they came with…

[NEW MODEL] SupraLabs just released a new model! - Supra-50M-Reasoning

SupraLabs just released a new model! - Supra-50M-Reasoning Hello again r/LocalLLaMA! Supra-50M-Reaso…

Bringing Gemma 4 12B to your Laptop: Unlocking Local, Agentic Workflows with Google AI Edge

submitted by /u/zxyzyxz [link] [comments]…

What is your current go-to stack for running a fully local AI agent?

Curious to know what quantization level (GGUF/EXL2) you find balances speed and smarts for daily use…