100 Trillion+ Pretraining data??? This is the largest data I've see a model being trained on.
https://preview.redd.it/oss7g2gnll4h1.png?width=894&format=png&auto=webp&s=5d4295707a700ed7541c274b8be8ad75bbd0903d Usually we see 27-50 Trillion tokens in most models, kimi, mimo, deepseek. They seem to have doubled the
📄
This source provides headlines only. Use the button below to read the complete article on the original site.
📰 Read the original article on r/LocalLLaMA
Originally published by r/LocalLLaMA. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.