Neural networks
What is a neural network made of?
A neural network is a set of connected units called artificial neurons. Each connection carries a weight, each neuron adds up its weighted inputs and passes the result through a function, and the neurons are arranged in layers from input to output.
A neural network is built from many small, simple units wired together. Each unit is called an artificial neuron (or a node), and the units are joined by connections. Every connection carries a number called a weight, which sets how strongly one neuron’s signal affects the next.[1]
A single neuron does something modest. It takes the outputs of the neurons feeding into it, multiplies each by the weight on that connection, and adds the results together. It adds one more number, called a bias. Then it passes that total through a function (a nonlinear activation function) to produce its own output.[1] The activation function matters more than it sounds: without it, stacking layers would gain you nothing, because a chain of plain linear steps is still just one linear step.[2]
The neurons are organized into layers. The first layer takes in the raw data. The last layer gives the answer. Any layers in between are called hidden layers, and they let the network recombine the data into more useful forms before reaching a result.[1, 2] What makes a network learn anything is the set of weights, and how they get set is the subject of the next page.
Where neural networks are built
- PyTorch ↗
A widely used open source machine learning framework for building and training neural networks.
- TensorFlow ↗
An end-to-end platform for machine learning, used to build and run neural networks.
- Keras ↗
A deep learning API that runs on top of JAX, TensorFlow, or PyTorch, aimed at readable code.
References
- Neural network (machine learning) — Wikipedia
- Neural networks: Nodes and hidden layers — Google for Developers