Let’s talk quants of Gemma and Qwen - 16 vs Q8 vs Q4 - any experiences?
Some people say they’d never go under Q8, and others say they find Q3 acceptable! What’s your take? submitted by /u/Borkato [link] [comments]
📄
This source provides headlines only. Use the button below to read the complete article on the original site.
📰 Read the original article on r/LocalLLaMA
Originally published by r/LocalLLaMA. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.