AI tools, cybersecurity and development news aggregated from top sources — saved permanently with unique URLs.
A First Comprehensive Study of TurboQuant: Accuracy and Performance
TL;DR from the article: FP8 via --kv-cache-dtype fp8 remains the best default for KV-cache quantiza…
llama.cpp constantly reprocessing huge prompts with opencode/pi.dev
I’m using llama-swap with llama.cpp. I mainly use opencode + pi.dev and I’m seeing frequent massive …
NVIDIA Reportedly Prepares RTX 5090 Price Hike Amid Rising GDDR7 Costs (maybe RTX 50 and PRO series as well)
submitted by /u/panchovix [link] [comments]…
Developing open source LLM from ground up from pretrain - rlhf(PPO/GRPO)
Hello I have been working on creating a LLM from ground up. It is based on deepseek architecture wit…
Is there a big gap between Q4 and Q6 on Qwen3.6?
I’ve got one 3090 and thanks to the help of MTP and all, I can do around 65 tok/s on qwen 3.6 dense …
Introducing cyankiwi AWQ 4-bit Quantization — 26.05 update
In standard AWQ, per-channel scales and quantization ranges are picked in separate steps: scales fir…
I tracked EU GPU prices across 15 stores for 50+ days - RTX 5090 is the only card not dropping in price
been tracking EU GPU prices since early march - 15 stores, 6-hour scrape cadence, ~126k readings. po…
Linux - Why does llama.cpp ROCm consume SO much VRAM for KV cache compared to Vulkan?
I have a docker stack with a bunch of AI services and llama.cpp server is the brain. I've got a work…
MLX 16/8/4/2-bit quants of nvidia/llama-embed-nemotron-8b
I converted nvidia/llama-embed-nemotron-8b to MLX fp16, 8-bit, 4-bit, and 2-bit (for my OCD) and put…
MOOSE-Star (ICML 2026): 7B model + 108K-paper dataset for scientific hypothesis discovery
Disclosure first: I work on community at MiroMind. One of our researchers just dropped the full MOOS…
The RTX 5000 PRO (48GB) arrived and it is better than I expected.
I posted here about buying it a few days ago: https://www.reddit.com/r/LocalLLaMA/comments/1t2slmw/f…
Is there a limit on the number of active parameters in an MoE model?
Hi. We recently had MoE models as big as 1T and 1.6T total parameters. My expectation on the proport…