Skip to content
All apps

Regression line: the derivation

About this app

Least squares step by step with lecture data (motor, mortgage rate, your own points): grab the line y = a + b·x (shift = a, turn = b), distances as real squares, their sum d²(a, b) as a 3D bowl. Sections with tangents show the partial derivatives, setting them to zero gives two lines in the (a, b) plane that cross at the minimum. The derivation down to b = s_xy/s_x² and a = ȳ − b·x̄ runs animated alongside, plus rolling a ball and centring x.

With explain mode: an AI tutor walks you through the app, points out what matters in the scene and answers your questions.

What it is about

The regression line y = a + b·x is the line for which the sum of vertical squared distances d²(a, b) = Σ(y_i − a − b·x_i)² is smallest. Differentiate d² partially with respect to a and b, set both derivatives to zero and you get the normal equations a + b·x̄ = ȳ and a·n·x̄ + b·Σx_i² = Σx_i·y_i. They give b = s_xy/s_x² and a = ȳ − b·x̄: the line passes through the centroid (x̄, ȳ).

What the app shows

On the left a scatter plot with data from a statistics lecture (power of a DC motor against speed, mortgage rate and construction orders) and a line you can shift and turn. The distances appear as real squares, their sum as a bar. On the right d²(a, b) stands as a bowl over the (a, b) plane. Sections at fixed b and at fixed a show the partial derivatives through their tangents, and the conditions “derivative = 0” appear as two lines that cross at the minimum. At the top the derivation runs alongside in twelve animated steps.

Try it yourself

Turn and shift the line and watch the ball travel on the bowl. Go through the derivation step by step: at “a = ȳ − b·x̄” the line jumps through the centroid, when the numbers are inserted it turns into the regression line, for the motor b = 0.012495 and a = 0.48726. Let the ball roll and centre x: the narrow valley becomes a round bowl. For the mortgage rate the line y = 4880 − 340x gives the estimate of €2670 m at 6.5 %.

Frequently asked questions

How do you calculate the regression line by least squares?
Minimise d²(a, b) = Σ(y_i − a − b·x_i)². Set the partial derivatives −2Σ(y_i − a − b·x_i) and −2Σ(y_i − a − b·x_i)·x_i to zero. This gives b = (Σx_i·y_i − n·x̄·ȳ)/(Σx_i² − n·x̄²) = s_xy/s_x² and a = ȳ − b·x̄.
Why does the regression line pass through the centroid?
The first normal equation comes from ∂d²/∂a = 0 and, divided by −2n, reads a + b·x̄ = ȳ. So at x̄ the line has exactly the height ȳ.
Why is the result a minimum?
d²(a, b) is a sum of squares and therefore a quadratic polynomial in a and b whose graph is an upward-opening bowl as long as not all x_i are equal. Such a surface has exactly one lowest point, and there both partial derivatives are zero.
What can I change in the app?
The data set (motor, mortgage rate, your own points), the line by handle or slider, the ball on the bowl, the points themselves, the derivation step, the view of the bowl and whether x is centred. Rolling the ball shows gradient descent, all sums and derivatives are in the Values panel.

Subject: Statistics

More from Statistics

A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps