AI tools, cybersecurity and development news aggregated from top sources — saved permanently with unique URLs.
Gemma 4 31B QAT GGUF loads with MTP branch, but outputs repeated <unused49> - any working recipe?
I’m trying to run: unsloth/gemma-4-31B-it-qat-GGUF gemma-4-31B-it-qat-UD-Q4_K_XL.gguf on an RTX 5090…
Open models to win ✌
submitted by /u/pmttyji [link] [comments]…
How to compare Original vs QAT Gemma 4 31B Q4 quants
I just came across the following post, where a user found some confusing divergence results between …
You don't need a GPU to run gemma-4-26B-A4B
I've been running LLMs on my old potato i5-8500 with 32GB of RAM and *no GPU* for awhile now, runnin…
Dense vs MoE quantization resiliance
Which one is more resiliant to quantization? Especially at 4-bit? My experience:i tried gemma4 26b a…
Cool stuff to do with NVIDIA RTX 6000 PRO 96GB VRAM
I have been a C++ dev for 3 years as long as have done PyTorch in my free time (not that good in the…
Alternatives to ChromaDB for easy RAG search
I'm disappointed that ChromaDB's local, free "single node" version is still getting second-class, ha…
GraphKV, kv cache optimization based on graph embedding models
I've been working on a project inspired by TurboQuant, It isnt perfect but it's pretty good for a pr…
5 Months Later: open-deepthink Now Has Full Knowledge Distillation Mode
Hey r/LocalLLaMA, Some of you might remember when I posted about this project back around September …
I can't wait for all the x250 sample distills of Mythos and GPT-5.6
Just kidding. Are there any distills that actually improve a model's quality? I remember the Qwen R1…
Gemma4 12B - Experiences?
Anyone check out the new Gemma4 12B that dropped 3 days ago? Integrated vision and audio recognition…
Gemma 4 31B QAT Q4 vs standard Q4 — Top1 KLD benchmark results have me confused. Someone please explain or poke holes in this.
I'll be upfront: I vibe-benched and vibe-reported this with Claude Sonnet 4.6, but I reviewed and ed…