r/LocalLLaMA

Drastically improve prompt processing speed for --n-cpu-moe partially offloaded models

Bigger ubatch made gpt-oss-120b prompt processing much faster on my RTX 3090 I was tuning gpt-oss-12…

Found a way to cool the DGX

Tap water keeps the temperature below 68 degree Celsius at 95% GPU utilization running Qwen3.5-122b-…

B9109: preemptive fix for mtp & mmproj fix soon? It appears so

Summary : spec : process images through the draft context — this directly addresses the mmproj + MTP…

I catalogued every way local models break JSON output and built a repair library, here's what I found across 288 model calls

I've been running structured output prompts through a bunch of models on OpenRouter for the past few…

prompt caching, but for rl training - 7.5x speedup on long-prompt/short-response workloads

most open source RL engines pack sequences naively: prompt + response, repeated for every sample in …

500k context on 48gb VRAM!! - 21tok/s (coding)

I found this model hiding in the corner of huggingface: https://huggingface.co/Max-and-Omnis/Nemotro…

Computer build using Intel Optane Persistent Memory - Can run 1 trillion parameter model at over 4 tokens/sec

As the title states, my build is indeed able to run a 1 trillion parameter model (in this case Kimi …

Will there be any more Qwen3.6 series models?

I'm still hoping we see a Qwen3.6-122B or a Qwen3.6-coder, but my hopes are dimming. Seems like we w…

PowerColor launches Radeon AI PRO R9600D with 32GB GDDR6 memory

https://videocardz.com/newz/powercolor-launches-single-slot-and-passive-radeon-ai-pro-r9600d-32gb-an…

Gemma 4 running fully offline on WebGPU with Transformers.js, controlling Reachy Mini over WebSerial.

submitted by /u/xenovatech [link] [comments]…

MiniCPM 4.6

submitted by /u/themrzmaster [link] [comments]…

Orc (working name) - auditable and declarative AI workflow

I’m building a small “Orchestration as Code” repo for LLM workflows. Does this concept make sense? I…