AI tools, cybersecurity and development news aggregated from top sources — saved permanently with unique URLs.
Gemma 4 31B's competence surprised me
I'm just getting started using local LLMs for code. I'm not interested vibe coding, but I am hoping …
Cheapest setup for >10 tok/sec for 120B dense LLM
Hi all, I'm trying to wrap my head around hardware variables when it comes to LLM, and I have anothe…
Have we reached the point where open-source LLMs are “just good enough”?
The question I’m asking myself is whether open-source LLMs are now “just good enough” to meet 95% of…
benchmark idea: political compass for finetuned/abliterated models
There are political compass benchmarks for cloud models, like this one:https://trackingai.org/politi…
Does CPU matter for GPU inference?
Hi, I'm currently building a PC which is exclusively going to be used for LLM inference. I'd like to…
Qwen 3.6 for coding with 5090 - Your settings recommendations?
Hi, totally new to using LLMs for coding purposes, I am on Ubuntu and currently using LM Studio with…
silx-ai/Quasar-Preview • Huggingface (5M context length)
https://huggingface.co/silx-ai/Quasar-Preview https://preview.redd.it/ur27udpzy66h1.jpg?width=900&fo…
Anyone seen benchmarks comparing Gemma 4 4-bit QAT vs. 8-bit standard quants?
I'm trying to find out if anyone has done any benchmarking comparing the Gemma 4 4-bit QAT models (v…
Gemma 4 26B A4B IT QAT Comparison
Hopefully this isn't too low effort of a post. I just finished the benchmarks and I figured I'd post…
ggml-webgpu: Improve prefill speeds for k-quants + refactor matmul for Q4/Q5/Q8 and k-quants by yomaytk · Pull Request #24225 · ggml-org/llama.cpp
This PR improves matmul performance for k-quants. The following table shows the improvement on the p…
2X tk/s (from 19.4 -> 38.1 tk/s on 1 x MI50) Playing with a hypothesis like speculative decoding.. but instead of an additional side model, exploiting that I can run multiple computations side-by-side AS IF I had Qwen3.6-27B loaded twice in memory - small quants don't use all the available compute.
Forgive the claude summary, in the readme, but the base works. I'm still working on the hip kernal a…
Jetbrains Mellum 2: a really good and performant model
Oh Hey Folks, I took the Mellum 2 model for a spin, so I wanted to share my impressions here. Discl…