AI tools, cybersecurity and development news aggregated from top sources — saved permanently with unique URLs.
What’s your most unusual non-LLM AI you actually use daily?
What’s your most unusual or underrated non-LLM AI tool you actually use daily (weird, niche, or non-…
llama.cpp Gemma4 MTP support merged!
submitted by /u/pinkyellowneon [link] [comments]…
llama : add Gemma4 MTP by am17an · Pull Request #23398 · ggml-org/llama.cpp
now your personal local Gemma will be faster than ever submitted by /u/jacek2023 [link] [c…
Need some guidance toying with local models
Hi, so I have a pretty low-end laptop regarding running LLMs locally (NVIDIA GeForce RTX 3050 with 4…
Qwen 3.6 27B KV cache quant benchmarks: 75 pairs, q8/q6/q5/q4, KVarN, Turbo/TCQ
Full benchmark results and in-depth analysis are available in the articles: KV Cache Quantization Be…
Dockerized Nemotron 3.5 ASR — Switched from Parakeet, better multilingual support + streaming (4.5x realtime speed on cpu)
I was originally using Parakeet for my speech recognition pipeline but decided to give Nemotron 3.5 …
How do you increase prompt processing speed ?
I am rocking Qwen like we all know, at 24GB 7900XTX 230k context, but it starts at 850t/s and then l…
Clustering 3x Jetson Nano Orin Supers
Hey everyone! Recently, I released a blog on how to setup a cluster out of your Raspberry Pi 4bs and…
Gemma 4 31B QAT GGUF loads with MTP branch, but outputs repeated <unused49> - any working recipe?
I’m trying to run: unsloth/gemma-4-31B-it-qat-GGUF gemma-4-31B-it-qat-UD-Q4_K_XL.gguf on an RTX 5090…
Open models to win ✌
submitted by /u/pmttyji [link] [comments]…
How to compare Original vs QAT Gemma 4 31B Q4 quants
I just came across the following post, where a user found some confusing divergence results between …
You don't need a GPU to run gemma-4-26B-A4B
I've been running LLMs on my old potato i5-8500 with 32GB of RAM and *no GPU* for awhile now, runnin…