Neural networks
How does a neural network learn?
A neural network learns by adjusting its weights. It makes a prediction, measures the error against the right answer, then sends that error backward through the network to nudge each weight toward a better result, and repeats this over many examples.
When a neural network is new, its weights are essentially random, so its answers are too. Learning is the process of fixing that. Put plainly, “training/learning involves adjusting the weights of the network to improve the accuracy of the result.”[1] Nothing about the network’s wiring changes; only the numbers on the connections do.
The loop is short and repeats many times. The network takes an example, runs it through to produce a guess, and that guess is compared to the known right answer to get an error. The bigger the error, the more the weights need to move. The usual method for figuring out which weights to move, and in which direction, is called backpropagation, which “spreads the error (adjusts the weights) from the output nodes across the network to the input nodes.”[1]
Each pass nudges the weights a little, so any single example barely shifts them. The learning comes from doing this across a great many examples until the errors get small. A network does not memorize a rule someone wrote for it. It settles, gradually, into a set of weights that happens to give good answers.
References
- Neural network (machine learning) — Wikipedia