Tag: chain-rule
All the articles with the tag "chain-rule".
Backpropagation, by Hand: The Chain Rule, Run Backward
Updated:Why a hidden layer can learn what a straight line can't, and how backpropagation gets the gradient of every weight in one backward pass. Worked by hand on XOR with every number, then as matrix rules for a whole layer, checked against finite differences, and costed: where the '6 × parameters FLOPs per training token' rule comes from.