What do people mean by a model, its parameters, and training?

A model is the trained program at the center of any AI system. A large language model, the kind behind chat assistants, is “a very large deep learning model that is pre-trained on vast amounts of data.”^[1] You can think of it as a single large mathematical function: text goes in, and a prediction about the next piece of text comes out.

A parameter is one of the numbers inside that function. Modern models have billions of them. In AWS’s words, “weights and biases along with embeddings are known as model parameters.”^[1] These numbers are not written by a person. They start out close to random and get shaped by the data the model sees. The count you hear quoted, like “70 billion parameters,” is roughly a measure of the model’s size and capacity.

Training is the process that sets those numbers. “During training, the model iteratively adjusts parameter values until the model correctly predicts the next token from the previous sequence of input tokens.”^[1] So training is not the model memorizing answers. It is a long loop of guessing the next bit of text, checking how wrong the guess was, and nudging the parameters to be a little less wrong next time, repeated across a huge amount of text.