Hypothesis test & p-value
About this app
A one-sided test made tangible: H₀ and H₁ distributions, the critical value from α, Type I error (α) and Type II error (β) as areas, the test power and the p-value — slide the effect size and sample size and watch the power rise.
The statistics
A hypothesis test uses a sample to decide between the null hypothesis H₀ and an alternative H₁. The significance level α determines a critical value c: if the test statistic lies beyond it, H₀ is rejected. A type I error (probability α) is rejecting a true H₀; a type II error (probability β) is keeping a false H₀. Power, 1 − β, is the probability of detecting an effect that is really there. The p-value is the probability, under H₀, of a result at least as extreme as the one observed.
What you see
The app shows a one-sided (right-tailed) test for the mean x̄ with standard error 1/√n. The H₀ distribution is centred on μ = 0, the H₁ distribution on the effect size μ₁. The critical value c splits the axis; the α area to the right of c under H₀ and the β area to the left of c under H₁ are shaded. A values panel lists c, α, β, power 1 − β and the p-value for the observed value you set, whose area can be shown on the chart.
Things to try
Increase the sample size n: both distributions get narrower, β shrinks and power rises even though the effect stays the same. Lower α from 0.05 to 0.01 and watch c move right while β grows; fewer false alarms cost power. Increase the effect size μ₁ and the distributions move apart. Move the observed x̄ and compare the p-value with α: whenever p is below α, x̄ lies beyond c.
Frequently asked questions
- What is a p-value?
- The p-value is the probability, assuming the null hypothesis is true, of observing a result at least as extreme as the one actually observed. A small p-value is evidence against H₀. It is not the probability that H₀ is true.
- What is the difference between a type I and a type II error?
- A type I error means rejecting a true null hypothesis, a false alarm; its probability is the significance level α. A type II error means keeping a false null hypothesis, a missed effect; its probability is β. For a fixed sample size, making one smaller makes the other larger.
- How can you increase the power of a test?
- Power 1 − β increases with a larger sample, a larger effect, less variability and a higher significance level α. In practice it is usually planned through the sample size. A power of 80 % is a common target in many fields.
- What can I set in the hypothesis test app?
- You set the significance level α, the effect size μ₁, the sample size n and the observed mean x̄. The app shows the critical value, α and β areas, power and p-value, and the p-value area can be switched on.
Subject: Statistics
More from Statistics
- Anscombe's quartet
- Bias-variance / overfitting
- Box plot workshop
- Geometric mean
- Law of large numbers
- Histogram workshop
- Confidence interval
- Correlation: why standardise?
A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps