AI tools, cybersecurity and development news aggregated from top sources — saved permanently with unique URLs.
sycl : port multi-column MMVQ from CUDA backend (~45% speculative decoding speedup on Intel Arc) by masonmilby · Pull Request #21845 · ggml-org/llama.cpp
Saw this on other sub so posting here. For Intel ARC card holders. Big boost so update llama.cpp ver…
PSA: Gemma 4 12B is NOT completely broken for coding and tool calling, you need a special chat template
This is a PSA for people like me who tried it and hit the wall with tool calls failing left and righ…
I built a iOS app to benchmark GGUF models on your iPhone/iPad
Hey I've been working on GenBench, a free iOS app that lets you download, run, and benchmark GGUF m…
Maybe KV cache offload to RAM isn't bad
So, llama.cpp has the -nkvo (--no-kv-offload) option to offload KV cache to RAM instead of VRAM. Man…
model: Granite4 Vision by gabe-l-hart · Pull Request #23545 · ggml-org/llama.cpp
Model Summary: Granite Vision 4.1 4B is a vision-language model (VLM) that delivers frontier-level p…
Gemma 4 QAT GGUFs from Unsloth
Their collection: https://huggingface.co/collections/unsloth/gemma-4-qat And their guide, always a v…
Gemma 4 with quantization-aware training
Google's collections: https://huggingface.co/collections/google/gemma-4-qat-q4-0 https://huggingface…
Unsloth just dropped MTP GGUF weights for Gemma 4!
It appears like Unsloth pushed MTP GGUF weights (Q8, F16, BF16) for 31B, 26B-A4B, 12B. https://huggi…
FYI llamacpp server can hot swap models now-a-days in under 30sec
See this question at least a handful of times when browsing new and in the comments, llamacpp has on…
Nemotron 3 Ultra is available on HuggingChat
impressive speed/performance ratio! served by togetherAI :) submitted by /u/paf1138 [link] …
Qwen 3.6-27B on vLLM with dual RTX 3090s: looking for launch parameters
Hi everyone. Please share your working launch commands for running Qwen 3.6-27B via vLLM on dual RTX…
I implemented KVarN in my llama.cpp fork and ran KLD benchmarks. It's promising!
Saw this post here yesterday: KVarN: new KV-cache quant from Huawei. 3–5× KV cache compression with …