AI tools, cybersecurity and development news aggregated from top sources — saved permanently with unique URLs.
Benchmark & Reality Check on Gemma 4 12B: Great model, but your local settings are probably breaking it (Fix inside)
I completed a Python bug hunting benchmark with Gemma 4 12B. I used the Unsloth Dynamic Q5 GGUF mode…
RTX Pro 4500 Blackwell Performance Numbers
RTX Pro 4500 Blackwell About one month ago I asked the fine people of Reddit for some upgrade advice…
Is there a quant of Granite 30b I can run in 12gb of VRAM/32gb of RAM?
I am hoping there is submitted by /u/MrMrsPotts [link] [comments]…
[Opinion] Gemma4-12B means that Google is going hard after the market of IoT and mobile and we're helping them
I know it might be a no-brainer in retrospect, but hear me out, y'all, it's not the whole story. [ti…
Gemma 4 12B is my new main squeeze
The Unsloth Q5_K_XL is officially my main squeeze for local coding. I started out with the Q4_K_XL, …
Can MTP models be used as standalone smaller models? (e.g. DS4 Flash/Pro)
I've been wondering about models that are trained with MTP (Multi-Token Prediction) and whether the …
PSA: You may not need to quantize spec draft when using MTP
Using `--spec-draft-type-k q4_0 --spec-draft-type-v q4_0` might actually decrease your context size!…
hello there! i made a tool to explore kokoro.
i built this on top of my own stack but the code is MIT for everything related. kokoro was pretty fu…
Here is my llama.cpp NVFP4/MXFP6 GGUF quantizer tool
Hello everyone I wanted to share what I've been working on. I started writing NVFP4 kernels for llam…
Magenta RealTime 2: Open & Local Live Music Models
Build and play AI musical instruments on your laptop! submitted by /u/phone_radio_tv [link] …
Finally finished my LLM server: EPYC 9575F, 4× RTX 3090 (96GB VRAM), 768GB ECC RAM
Took a while, but Nalthis is finally up and assembled. Specs: Supermicro H13SSL-N AMD EPYC 9575F (6…
Kimi K2.6 on 8×B200: expected vLLM/SGLang throughput?
I’m planning to run moonshotai/Kimi-K2.6 on 8×NVIDIA B200 with vLLM or SGLang, likely using NVFP4(or…