Survivorship bias
About this app
Abraham Wald’s bombers: fly missions, analyse only the survivors and reveal the losses — with an armor comparison, Wald’s inference of vulnerability and a fund database as transfer.
The idea
Survivorship bias arises when only cases that made it through a selection process are analysed. The best-known example comes from World War II: at the Statistical Research Group at Columbia University, the statistician Abraham Wald was asked to estimate how vulnerable aircraft were in different places. His data came only from planes that had returned. Where survivors show surprisingly few hits, the planes hit there are exactly the ones missing, because they went down. Few holes therefore point to a vulnerable spot. The widely shared aircraft graphic with red dots is a modern illustration.
What you see
In the model each plane takes λ hits on average, Poisson-distributed and spread evenly over its area. Each region has its own probability that a hit brings the plane down, for example 75 % for nose and cockpit and 4 % for the wings. You fly missions with 50, 300 or 1000 planes and switch between “Survivors only” and “With losses”. The analysis panel compares observed with expected holes and, following Wald’s idea, estimates λ̂ = H̄ − ln Ŝ and the vulnerability q̂ of each region from the survivors and the number of losses alone.
Things to try
Place up to two armor plates and compare the strategies “Where the holes are” and “Where there are no holes”. Each strategy is simulated with 2000 planes, and a bar shows the exact return rate P = e^(−λ·Σ aᵣqᵣ). Watch which strategy brings more planes home. The funds panel shows the same trap in finance: 200 funds without skill run for ten years, and any fund that drops below 85 % of its starting value is closed. The average of the remaining funds then looks like skill, although the true excess return of every fund is zero.
Frequently asked questions
- What is survivorship bias?
- Survivorship bias is a distortion that occurs when a data set contains only the “survivors” of a selection process. Failed, closed or lost cases are missing, so the analysis gives a picture that is too positive or otherwise skewed. Typical examples are success stories without the failures and fund statistics without closed funds.
- What did Abraham Wald find out about the planes?
- In 1943 memoranda for the Statistical Research Group, Wald developed a method to estimate aircraft vulnerability from the damage on planes that returned. The key insight: places where survivors show few hits are dangerous, because planes hit there did not come back. The popular armor anecdote is a simplified version of this work.
- How does survivorship bias distort fund returns?
- When poorly performing funds are closed and removed from a database, only the more successful ones remain. Their average lies above that of all funds ever launched. Empirical studies of mutual funds find biases on the order of about one percentage point per year.
- What can I do in this app?
- You fly missions, show or hide the losses and place up to two armor plates. A fleet comparison shows which armor strategy brings more planes home, and the analysis estimates each region’s vulnerability from the survivors only. The funds panel transfers the idea to fund databases.
Subject: Data & decisions
More from Data & decisions
- Anchoring effect
- Benford's law
- Berkson's paradox
- Cherry-picking: base period
- Cherry-picking: base period 2.0
- The three-cowboy duel (truel)
- The three-cowboy duel 2.0
- The three means
A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps