Skip to content
All apps

Curve-fitting playground

About this app

A model is a form with parameters: polynomials of degree 0 to 12 by least squares, residuals as squares (area = loss), held-out error over the degree and parameters that explode when overfitting.

Models as shapes with knobs

A regression model is a shape with parameters: a line has two, a polynomial of degree d has d + 1. Least squares chooses the parameters so that the sum of squared distances between data points and curve is as small as possible. As the degree goes up, this training error keeps falling. Whether the model is any good only shows on data that was not used for fitting. A shape that is too simple underfits; one that is too flexible and also learns the noise overfits.

What you see

Three cards work together. The data card shows training points, held-out points, the fitted curve, optionally the true function, and the errors as actual squares. The error card plots training and held-out RMSE against the degree and marks the best degree. The parameter card shows the coefficients on a log scale; when the model overfits they become very large. You can choose a line, a parabola or a polynomial up to degree 12, plus ridge smoothing λ and three noise levels.

Things to try

Raise the degree step by step and watch both error curves: the training error keeps dropping, while the held-out error first falls and then rises again. At high degrees the curve swings wildly between the points. Then increase the smoothing λ and see the swings and the coefficients shrink. You can place training points with a click, drag them, delete them with a double click, or draw a fresh sample.

Frequently asked questions

What is overfitting?
Overfitting means a model matches its training data very closely but performs poorly on new data. Besides the actual pattern it has also learned the random noise. The typical sign is a small training error combined with a large error on held-out data.
What is the difference between underfitting and overfitting?
An underfitting model is too simple to capture the pattern, so both training and test error are high. An overfitting model is too flexible and adapts to the noise, so the training error is low and the test error high. A good model complexity lies in between, where the test error is lowest.
How does ridge regression reduce overfitting?
Ridge regression adds a penalty λ · Σaⱼ² to the sum of squared errors, making large coefficients expensive. The parameters shrink and the fitted curve becomes smoother. With λ = 0 you get the ordinary least squares solution.
What can I try in this app?
You choose a line, a parabola or a polynomial up to degree 12, set the noise and the ridge smoothing, and place or drag data points. The app shows the squared errors, training RMSE and R², and the error on held-out points. Clicking in the error card selects a degree directly.

Subject: Probability & statistics

More from Probability & statistics

A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps