The qlora 2e-4 default is wrong under 10k samples and nobody talks about it [D]
Every qlora tutorial on earth says start at 2e-4. Unsloth docs, hf examples, the paper itself. and for small datasets i now think that numbers is a trap. Where does 2e-4 come from? alpaca. 52k samples. cool, except most
π
This source provides headlines only. Use the button below to read the complete article on the original site.
π° Read the original article on r/MachineLearning
Originally published by r/MachineLearning. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.