AI tools, cybersecurity and development news aggregated from top sources — saved permanently with unique URLs.
Initial testing with llama-bench and 3 different Qwen3 models for my R9700 32GB
In a recent build I did I used dual R9700 32GB cards but I wanted to see how a single R9700 stacked …
What exactly is quantization aware training?
First time hearing it. I also heard about the gemma 4 qat quants and if any one of them is good for …
Don’t act like y’all ain’t thinking it. I’m just saying the quiet part out loud. /s
Of course I’m thankful for all that Qwen has bequeathed us, but deep down in the darkest pit of our …
At least one more Gemma 4 model confirmed??
submitted by /u/Sufficient-Bid3874 [link] [comments]…
sycl : port multi-column MMVQ from CUDA backend (~45% speculative decoding speedup on Intel Arc) by masonmilby · Pull Request #21845 · ggml-org/llama.cpp
Saw this on other sub so posting here. For Intel ARC card holders. Big boost so update llama.cpp ver…
PSA: Gemma 4 12B is NOT completely broken for coding and tool calling, you need a special chat template
This is a PSA for people like me who tried it and hit the wall with tool calls failing left and righ…
I built a iOS app to benchmark GGUF models on your iPhone/iPad
Hey I've been working on GenBench, a free iOS app that lets you download, run, and benchmark GGUF m…
Maybe KV cache offload to RAM isn't bad
So, llama.cpp has the -nkvo (--no-kv-offload) option to offload KV cache to RAM instead of VRAM. Man…
model: Granite4 Vision by gabe-l-hart · Pull Request #23545 · ggml-org/llama.cpp
Model Summary: Granite Vision 4.1 4B is a vision-language model (VLM) that delivers frontier-level p…
Gemma 4 QAT GGUFs from Unsloth
Their collection: https://huggingface.co/collections/unsloth/gemma-4-qat And their guide, always a v…
Gemma 4 with quantization-aware training
Google's collections: https://huggingface.co/collections/google/gemma-4-qat-q4-0 https://huggingface…
Unsloth just dropped MTP GGUF weights for Gemma 4!
It appears like Unsloth pushed MTP GGUF weights (Q8, F16, BF16) for 31B, 26B-A4B, 12B. https://huggi…