How is it possible that OKLS can't outperform Muon in terms of step efficiency? [D]
SOAP is more step efficeint than Shampoo, and KL-Shampoo is more step efficient than SOAP. Then how is OKLS getting outperformed by Muon in terms of step efficiency where it's a zero-staleness implementation of KL-Shampo
π
This source provides headlines only. Use the button below to read the complete article on the original site.
π° Read the original article on r/MachineLearning
Originally published by r/MachineLearning. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.