Skip to content
All apps

How many answers are enough?

About this app

One workforce, many surveys: see how far results jump with few answers and why it takes four times as many answers to halve the uncertainty.

With explain mode: an AI tutor walks you through the app, points out what matters in the scene and answers your questions.

Why samples vary

A survey never hits the true value exactly. Every random sample gives a slightly different result, and a 95 % confidence interval states the range in which the true proportion plausibly lies. For a proportion p̂ it is roughly p̂ ± 1.96 · √(p̂(1 − p̂)/n). The width therefore shrinks with 1/√n: halving the uncertainty takes four times as many answers. The size of the population hardly matters as long as only a small part of it is surveyed.

What the app shows

A workforce of 10,000 people is asked about their satisfaction. Each survey appears as a line with its uncertainty range; one click repeats 100 surveys and shows how many cover the true value. A curve plots the uncertainty against the number of answers. In “two departments” mode, two equally satisfied departments are compared: how often does one still look clearly better? Intervals follow Wilson, and Newcombe for the difference.

Things to try

Set n to 20 and draw a few surveys, then quadruple n to 80 and 320. At 50 % satisfaction the uncertainty falls from about ±20 to roughly ±11 and just over ±5 percentage points. In comparison mode the app counts false alarms: differences that look real but are pure chance. Keep in mind that more answers only help with random selection, not against bias from voluntary participation.

Frequently asked questions

How many respondents does a survey need?
It depends on the precision you need. For a proportion around 50 %, 100 randomly selected answers give a 95 % margin of error of about ±10 percentage points, 400 answers about ±5 points. The size of the population is usually secondary.
Why does it take four times the sample to halve the margin of error?
The standard error of a proportion is proportional to 1/√n. Doubling n only reduces it by a factor of √2, about 29 %. Only a fourfold n halves the width of the confidence interval.
When is a difference between two survey groups meaningful?
A difference is only reliable when its confidence interval does not include zero. With small groups, gaps of many percentage points easily arise by chance alone. Even at 95 % confidence, roughly one comparison in twenty will be wrong.
What can I try in the app?
You draw single surveys or 100 repeats, change the number of answers with a slider or the steps 20, 80 and 320 and can show or hide the true value. In “two departments” mode you compare two equally satisfied groups. A formula panel works through the interval for the current survey.

Subject: Data & decisions

More from Data & decisions

A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps