What a parameter is — Physea Wiki

A parameter is one adjustable number inside the model. Training nudges all of these numbers until the model predicts text well, so the full set of parameters is where what the model learned actually lives.

A parameter is a single number stored inside a model. Most parameters are weights, and a weight is the strength of a connection between two of the model’s internal units. As one glossary puts it, each weight “signifies the strength and direction (positive or negative) of the influence one neuron has on another.”^[1] A second, smaller kind of parameter is the bias, a number added on top that shifts the result.

These numbers are not written by a programmer. They start out random and are adjusted during training. The network “learns by iteratively adjusting these weights to predict the correct output,” and “the set of weights in the network encapsulates what the network has learned from the training data.”^[1] So when people say a model “knows” something, that knowledge is not stored as text or rules. It is spread across the values of its parameters.

When a model finishes training, those values are frozen and saved to a file. That file is the model. Running it means reading the parameters back and doing arithmetic with them. The count of parameters is how a model’s size is usually reported, which is why model names so often end in a number followed by a “B” for billion.^[2]

In short A parameter is one learned number. Weights set the strength of connections, biases shift the result, and the whole collection of them is what the model learned.

References

Weight (Artificial Neural Network) — DeepAI Machine Learning Glossary
Large language model — Wikipedia

What is a parameter in a language model?

References