Scaling to 1T MoE on TorchTitan [P]
Hi everyone! Over the last few months we've been in the trenches with the TorchTitan and PyTorch maintainers, trying to find out how far GH200 (the GPUs inside the JUPITER supercomputer) can be pushed. Short answer: far
π
This source provides headlines only. Use the button below to read the complete article on the original site.
π° Read the original article on r/MachineLearning
Originally published by r/MachineLearning. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.