Mutating Gemma 4 31B Dense in to a native Gemma 4 additive-MoE model
I recently came across an interesting model on Hugginface from JDONE-Research/AIOne-Agent-52B-A36B-it. It is the first finetune I saw that is built on the Gemma 4 31B dense model but enables MoE for it, training a router
📄
This source provides headlines only. Use the button below to read the complete article on the original site.
📰 Read the original article on r/LocalLLaMA
Originally published by r/LocalLLaMA. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.