What does the 7B or 70B in a model name mean?

When you see a name like Llama 2 7B or Llama 2 70B, the B stands for billion and the number counts the model’s parameters. So 7B is about 7 billion parameters and 70B is about 70 billion. Meta released Llama 2 in exactly these sizes: 7B, 13B, and 70B.^[1]

A parameter is a single adjustable number inside the model. During training, the model nudges these numbers up and down so that its outputs better match the text it is learning from. A finished model is, in a literal sense, just a very large list of these settled numbers. When people loosely call them “weights,” they mean the same thing.

The count is a rough measure of how much the model can store and how much computation each answer takes. It is not a quality score on its own. A 70B model has roughly ten times as many parameters as a 7B model, but that does not make it ten times better, and the relationship between count and capability is the subject of the next page.