Using Q38FN with dflash2? Try lower draft model quants.
Back with Qwen 3.8 27B i heard the advice that a Q4 draft model would produce roughly the same acceptance rate as a Q8 or higher. I have had my agent create a Q38FN Draft model in F16, Q8 and Q4, and the Q4 seemed to del
๐
This source provides headlines only. Use the button below to read the complete article on the original site.
๐ฐ Read the original article on r/LocalLLaMA
Originally published by r/LocalLLaMA. Aggregated on AIWithGhost for educational purposes โ full credit and traffic to the original publisher.