I implemented BatchNorm, LayerNorm, and GroupNorm from scratch to see what they actually do to neuron activations [D]
I kept seeing BatchNorm/LayerNorm in model definitions without really understanding the assumptions behind each one, so I implemented all three (batch, layer, group) from scratch and ran them side by side on a simple 3-l
📄
This source provides headlines only. Use the button below to read the complete article on the original site.
📰 Read the original article on r/MachineLearning
Originally published by r/MachineLearning. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.