Maximum likelihood estimation
About this app
Fit a distribution to data: deform the model density, see the likelihood contributions of the points, follow the log-likelihood curve with its maximum (the MLE). The foundation behind regression, logistic regression and Naïve Bayes.
The statistics
Likelihood is not the same as probability: for probability the parameters are fixed and we ask about the data; for likelihood the data are fixed and we read the density as a function of the parameters. Maximum likelihood estimation looks for the parameters θ under which the observed data are most plausible: L(θ) = ∏ f(xᵢ; θ). In practice one maximises the log-likelihood ℓ(θ) = Σ log f(xᵢ; θ), whose maximum lies in the same place. For the normal distribution this gives μ̂ = x̄ and σ̂² = (1/n) Σ(xᵢ − x̄)².
What you see
On the left the data points sit under a model density N(μ, σ) that you can reshape. Vertical sticks show the density at each point, its contribution to the likelihood. On the right the log-likelihood runs as a curve over μ with σ fixed or over σ with μ fixed, with the maximum marked. A values panel lists the current parameters, ℓ(μ, σ), the estimates μ̂ = x̄ and σ̂ and the gap to the maximum, ℓ(MLE) − ℓ.
Things to try
Move μ away from the data and watch the sticks at distant points almost vanish as ℓ drops. Shrink σ sharply: a few points get high densities but the outliers get tiny ones, and ℓ falls. Work your way to the maximum until the gap to ℓ(MLE) reaches zero; you land at x̄ and σ̂. Draw new data or change the data set size n, and jump straight to the optimum with the set-to-MLE button.
Frequently asked questions
- What is maximum likelihood estimation?
- Maximum likelihood estimation estimates the parameters of a statistical model by choosing the values under which the observed data are most probable. It maximises the likelihood function or its logarithm. Many methods, including linear and logistic regression, can be derived this way.
- What is the difference between likelihood and probability?
- A probability describes, for fixed parameters, how plausible different data are. Likelihood reverses the view: the data are fixed and you assess how well different parameter values fit them. The likelihood is not a probability distribution over the parameters.
- Why use the log-likelihood?
- The likelihood is a product of many small densities and quickly becomes numerically tiny. Taking the logarithm turns it into a sum that is easier to differentiate and compute. Because the logarithm is strictly increasing, the maximum stays in the same place.
- What can I set in the MLE app?
- You set the model mean μ and the model spread σ, choose whether the log-likelihood is plotted over μ or over σ, change the data set size n, draw new data and can set the parameters straight to the MLE.
Subject: Statistics
More from Statistics
- Anscombe's quartet
- Bias-variance / overfitting
- Box plot workshop
- Geometric mean
- Law of large numbers
- Histogram workshop
- Hypothesis test & p-value
- Confidence interval
A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps