AI tools, cybersecurity and development news aggregated from top sources — saved permanently with unique URLs.
KVarN: new KV-cache quant from Huawei. 3–5× KV cache compression with actual speed-up instead of slow-down, and unlike TurboQuant it holds up on reasoning (Apache 2.0, vLLM single flag)
The KV-cache quant race just got more interesting. Huawei just open-sourced KVarN, a KV-cache quanti…
Need some help from someone who knows llama-cpp vulkan builds (docker in this case)
This morning I noticed Gemma4 31b's reasoning phase was being completely skipped. Confused, I starte…
I Built a Practical Guide to LLM Engineering: RAG, Retrieval, Rerankers, and Evaluation
If you’re building LLM apps and feel confused about when to use keyword search, embeddings, reranker…
Qwen 3.6 27B released 20 days after its plus announcement, 3.7 27B in 10th June?
Wondering if we will ever continue to get such strong models releasing, given that these little boys…
AMD & Intel, now onwards it's your turn to release your own models
What are you doing AMD & Intel? NVIDIA just released a 550B model after so many tiny/small/medium/bi…
What is your experience between Qwen3.6 27B at IQ3 and 35B-A3B at Q4?
If you’ve had the opportunity to compare these two together with your own benchmarks and use cases, …
Gemma 4 12B: incompatible with opencode, or just awful at tool calling?
Yesterday I tried out Gemma 4 12B on a significant coding challenge, to compare it to prior results …
Quick numbers on a BC250
Here is what I got on my BC250 with a fresh Llama-cpp (Vulcan) yesterday : - Fedora 44 - Ran stock, …
mistral.rs support for Gemma 4 12B - multimodal, agentic, and MTP integration
mistral․rs provides web search and safe, sandboxed code execution functionality to allow you to buil…
nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Hugging Face
Model Summary Total Parameters 550B (55B active) Architecture LatentMoE - Mamba-2 + MoE + Atten…
nex-agi/Nex-N2-mini • Huggingface
https://huggingface.co/nex-agi/Nex-N2-mini submitted by /u/External_Mood4719 [link] [comme…
Gemma 4 QAT confirmed to release soon!
It seems like this comment has gone widely unnoticed. https://old.reddit.com/r/LocalLLaMA/comments/1…