Skip to content
All apps

Neural Network — 3D Lab

About this app

Training lab for neural networks in 3D: neurons as tiles showing their output, edges by weight, spiral, XOR, moons, layers, activations and regularisation live; plus a digits mode.

How a network separates classes

A neuron in the first layer can only draw a straight boundary in the input plane. Every further layer combines these simple boundaries into curved ones, producing circles, arcs or spirals. Without a nonlinearity the whole network stays a single straight line, however deep it is. Learning works by gradient descent: backpropagation computes how each weight affects the error (cross-entropy), and all weights move a little in the better direction. L1 or L2 regularisation penalises large weights and counters overfitting.

What the app shows

Every neuron is a tile in a 3D scene. Its image shows what the neuron outputs at every point of the input plane; the lines between tiles are weights, with thickness for size and colour for sign. On the right lies the decision surface of the whole network, optionally as a 3D height field. A chart tracks training and test loss over the epochs, and a panel shows the weight matrices W and biases b. In digit mode the network reads the 15 pixels of a 3 × 5 grid.

Things to try

Try the spiral with one hidden layer, which fails, and then with three layers, which succeed. Choose up to four layers, the number of neurons per layer and ReLU, tanh, sigmoid or linear as activation. Add input features such as x₁², x₁x₂ or sin x₁: with the right features even a small network separates the circle. With noise up to 50 % and a large network, the test loss shows how overfitting develops.

Frequently asked questions

Why does a neural network need several layers?
A single neuron can only draw a straight boundary. Only when several layers with nonlinear activation are stacked can curved boundaries, such as those of a spiral, be represented. Each layer combines the simple boundaries of the previous one into more complex shapes.
What is backpropagation?
Backpropagation is a method that computes for every weight how strongly it affects the network’s error. It applies the chain rule to pass the error back layer by layer from the output to the input. Gradient descent then changes the weights a little in the direction that reduces the error.
What is a “dead” ReLU neuron?
A ReLU neuron outputs max(0, z). If its weighted sum z is below 0 for every input, it outputs 0 everywhere and no longer receives a gradient. It then contributes nothing to the result and stops learning.
What can I try in this app?
You choose dataset, noise, input features, layers, neuron count, activation, learning rate and regularisation and watch the network learn live. Every neuron can be clicked, and in digit mode you draw on a 3 × 5 grid yourself.

Subject: Neural networks

More from Neural networks

A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps