Skip to content
All apps

Neural Network

About this app

A neural network as a function: every neuron visible, ReLU kinks as building blocks of the output, weights editable per neuron, parameter count; curve, plane and digits modes.

A network is a function

A neural network takes numbers in and gives numbers out. Every neuron computes the same thing: a weighted sum of its inputs plus a bias, z = w₁a₁ + w₂a₂ + … + b, followed by an activation such as ReLU(z) = max(0, z). With ReLU, every neuron in the first layer is a kink, and the output is a sum of such building blocks, a curve made of straight pieces. A second layer bends these curves again. Without an activation, a network of any depth stays a straight line.

What the app shows

The app draws the network with every neuron; line thickness shows the size of a weight, colour its sign. In curve mode the network learns a target function such as a wave, bump, step or V shape, and the building blocks of the first layer are visible one by one. In surface mode it separates two classes in the plane, and in digit mode it recognises digits on a 3 × 5 grid with a softmax output. A counter shows the parameters: one weight per connection, one bias per neuron.

Things to try

Choose 0 to 3 hidden layers with up to 10 neurons each, and ReLU, tanh, sigmoid or no activation. Click a neuron: its card shows the calculation, and the neuron panel makes its weights adjustable. “Forward pass” sends an input through the network layer by layer, and “Fit” searches for matching weights by gradient descent. Set the activation to none and watch the output turn into a straight line.

Frequently asked questions

What does a neuron in a neural network compute?
A neuron multiplies each input by a weight, adds everything up together with a bias and applies an activation function. The result goes to the neurons of the next layer. Learning means adjusting weights and biases so that the output matches the target.
Why does a neural network need an activation function?
Without an activation function every layer is a linear map, and several linear layers in a row are again a linear map. The network could then only represent lines or planes. Only the nonlinearity, such as ReLU, makes kinks and curves possible.
How many parameters does a neural network have?
Every connection between two neurons has a weight, and every neuron except the inputs has a bias. A network with 1 input, 8 hidden neurons and 1 output therefore has 8 + 8 weights and 8 + 1 biases, 25 parameters in total.
What can I try in this app?
You set layers, neuron count and activation, adjust individual weights and see how the output changes. In curve, surface and digit modes you let the network learn by gradient descent and follow loss and accuracy.

Subject: Neural networks

More from Neural networks

A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps