AI tools, cybersecurity and development news aggregated from top sources — saved permanently with unique URLs.
16B dense on 16GB GPU vs 32B dense on 2x 16GB GPU
I'm currently trying to plan a build to run big(-ish) LLMs locally, and was wondering the following:…
Me: Arguing with an AI bot who just posted something on this sub about Llama 3.1.
For real tho, these bots need to turn on their web search functions and quit living in the past. It’…
Qwen3.6-35B-A3B tool calling benchmark: ByteShape vs. Unsloth GGUFs, KV cache quants & long context performance
I've previously posted some small performance benchmarks, but this time I got interested in the qual…
Was BitNet a dead end? What happened to ternary LLMs?
They seemed so promising at one point but the biggest ternary model is still 2B. What happened? Why …
LocalLLaMA post tier list
Since there is much (justified) whining about post quality, I thought it would be helpful to get a s…
When every other post is an AI generated benchmark report, a question about the best model, or a slop-coded application or engine that pretends to be groundbreaking
submitted by /u/Honest-Kangaroo-1830 [link] [comments]…
How-to guide to create audiobooks?
There are a number of projects posted in this sub aiming to convert ePub or RTF files to MP3, or jus…
Friends from the localllama community, if you love local llm, don't participate in the IPO (spaceX, OpenAI, Anthropic)
I'm not going to. And you shouldn't either. The frontier labs are the ones who are harming our comm…
An Implementation of NanoQuant: A flexible binary quantization method
https://github.com/pitbox46/NanoQuant TLDR: NanoQuant is a quantization method to create 2 bit/weigh…
Here are some tips on hitting nearly 200 tok/s for DeepSeek v4 Flash on Hopper
I needed a smarter model for my local Hermes Agent setup, so I moved to DeepSeek v4 Flash. First thi…
Why is the MLX version of the Gemma 4 QAT so big??
the MLX version of the QAT 4bit is like 27gb but the none QAT version is 17gb and the regular 4bit M…
I bundled a fully local LLM inside my Unity game. No internet, no cloud, no API key. The conversation is the gameplay.
I am making a game that is bundled with a local LLM and every conversation is unique. The game, 'Sim…