Logistic regression
About this app
Two features, a linear score, a sigmoid surface in 3D: the threshold plane cuts it in a straight line, log loss per point, gradient descent step by step, XOR as the model's limit.
The statistics
Logistic regression is a classification method for two classes. From features x₁ and x₂ it forms a linear score z = w₁x₁ + w₂x₂ + b. The sigmoid p = σ(z) = 1/(1 + e^−z) turns it into a probability for class 1. The decision boundary is where p reaches the threshold p*; in feature space it is a straight line. Training minimises the log loss L = −[y ln p + (1 − y) ln(1 − p)], usually by gradient descent. Classes that no straight line can separate, as in XOR, are beyond the model.
What you see
The sigmoid surface sits in 3D above the feature plane. Class 0 points lie on the floor, class 1 points at height p = 1. A threshold plane at p* cuts the surface along the decision boundary. Vertical sticks can show the loss for each point. A training panel runs gradient steps one at a time or continuously and reports the step count, mean log loss and gradient size; a model panel shows w₁, w₂, b and accuracy. You can view the scene at an angle, from above or in profile.
Things to try
Load the separable data set, reset the weights to 0 and press step a few times: the surface rises, the boundary turns between the classes and the log loss falls. Increase the learning rate η and compare how fast training converges. In the outlier data set, move single points and see how strongly they pull the boundary. Shift the threshold p* to see that only the boundary’s position changes. With XOR the model finds no suitable line.
Frequently asked questions
- What is logistic regression?
- Logistic regression estimates the probability that a data point belongs to one of two classes. It combines the features linearly into a score and converts it into a probability between 0 and 1 with the sigmoid function. Despite its name, it is a classification method.
- What is log loss?
- Log loss, or cross-entropy, scores predicted probabilities: L = −[y ln p + (1 − y) ln(1 − p)]. When the model is confident and right the loss is close to 0; when it is confident and wrong the loss becomes very large. Logistic regression picks the weights that minimise the mean log loss.
- Why can’t logistic regression solve XOR?
- The decision boundary of logistic regression is a straight line, or hyperplane, in feature space. In XOR the classes sit diagonally opposite each other and no line separates them. Extra features such as the product x₁·x₂, or a nonlinear model like a neural network, are needed.
- What can I try in the app?
- You place points of both classes, choose data sets (separable, overlapping, with outlier, XOR), set the threshold p* and learning rate η and train step by step or automatically. The app shows the sigmoid surface, the decision boundary, the loss per point, the weights and the accuracy.
Subject: Statistics
More from Statistics
- Anscombe's quartet
- Bias-variance / overfitting
- Law of large numbers
- Hypothesis test & p-value
- Confidence interval
- Measures of location & spread
- Measures of location & spread 2.0
- Maximum likelihood estimation
A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps