How many answers are enough?
About this app
One workforce, many surveys: see how far results jump with few answers and why it takes four times as many answers to halve the uncertainty.
With explain mode: an AI tutor walks you through the app, points out what matters in the scene and answers your questions.
Why samples vary
A survey never hits the true value exactly. Every random sample gives a slightly different result, and a 95 % confidence interval states the range in which the true proportion plausibly lies. For a proportion p̂ it is roughly p̂ ± 1.96 · √(p̂(1 − p̂)/n). The width therefore shrinks with 1/√n: halving the uncertainty takes four times as many answers. The size of the population hardly matters as long as only a small part of it is surveyed.
What the app shows
A workforce of 10,000 people is asked about their satisfaction. Each survey appears as a line with its uncertainty range; one click repeats 100 surveys and shows how many cover the true value. A curve plots the uncertainty against the number of answers. In “two departments” mode, two equally satisfied departments are compared: how often does one still look clearly better? Intervals follow Wilson, and Newcombe for the difference.
Things to try
Set n to 20 and draw a few surveys, then quadruple n to 80 and 320. At 50 % satisfaction the uncertainty falls from about ±20 to roughly ±11 and just over ±5 percentage points. In comparison mode the app counts false alarms: differences that look real but are pure chance. Keep in mind that more answers only help with random selection, not against bias from voluntary participation.
Frequently asked questions
- How many respondents does a survey need?
- It depends on the precision you need. For a proportion around 50 %, 100 randomly selected answers give a 95 % margin of error of about ±10 percentage points, 400 answers about ±5 points. The size of the population is usually secondary.
- Why does it take four times the sample to halve the margin of error?
- The standard error of a proportion is proportional to 1/√n. Doubling n only reduces it by a factor of √2, about 29 %. Only a fourfold n halves the width of the confidence interval.
- When is a difference between two survey groups meaningful?
- A difference is only reliable when its confidence interval does not include zero. With small groups, gaps of many percentage points easily arise by chance alone. Even at 95 % confidence, roughly one comparison in twenty will be wrong.
- What can I try in the app?
- You draw single surveys or 100 repeats, change the number of answers with a slider or the steps 20, 80 and 320 and can show or hide the true value. In “two departments” mode you compare two equally satisfied groups. A formula panel works through the interval for the current survey.
Subject: Data & decisions
More from Data & decisions
- Anchoring effect
- Benford's law
- Berkson's paradox
- Cherry-picking: base period
- Cherry-picking: base period 2.0
- The three-cowboy duel (truel)
- The three-cowboy duel 2.0
- The three means
A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps