Conclusions, errors and power

About this lesson

Statistics (AP-aligned) › Significance tests

To save a PDF, choose “Save as PDF” in the print dialog.

Conclusions, errors and power

Warm-up

Three quick questions from earlier lessons.

Discover

At α = 0.05 a test gives P-value 0.30. Does this prove H₀ is true?

  1. No: we just lack evidence against it
  2. Yes

Key idea

Decide, and know what can go wrong.

If P-value ≤α\le \alpha, reject H₀ (convincing evidence for Hₐ); otherwise fail to reject H₀. Type I error: rejecting a true H₀ (probability α). Type II error: failing to reject a false H₀. Power = probability of correctly rejecting a false H₀; it grows with nn and with α.

Worked example

H₀: a new teaching method is no better.

Type I: adopting a method that is not better
Reject a true H₀.
Type II: keeping the old method when the new one is better
Miss a real improvement.
Higher power, fewer Type II errors
More students in the study.

Your turn, guided

Your turn: α = 0.01. What is the probability of a Type I error when H₀ is true?

0.010.01
It equals α.

Practice ★

  1. At significance level α = 0.05, a test gives P-value 0.04. What do you conclude?
    1. Reject H₀: there is convincing evidence for Hₐ
    2. Accept H₀
    3. Hₐ is proven true
  1. A test of a new drug (sample of 388) has H₀: the drug has no effect. What is a Type I error?
    1. Concluding the drug works when it really has no effect
    2. Concluding there is no effect when the drug really works
    3. Making a mistake in the data

Watch out!

Practice ★★

  1. How can a researcher increase the power of a test that now uses 82 subjects?
    1. Use a larger sample
    2. Use a smaller α
    3. Use fewer subjects

Exit ticket

Three questions to check you have got it.

  1. At significance level α = 0.05, a test gives P-value 0.04. What do you conclude?
    1. Reject H₀: there is convincing evidence for Hₐ
    2. Accept H₀
    3. Hₐ is proven true
  2. At significance level α = 0.05, a test gives P-value 0.03. What do you conclude?
    1. Reject H₀: there is convincing evidence for Hₐ
    2. Accept H₀
    3. Hₐ is proven true
  3. At significance level α = 0.05, a test gives P-value 0.02. What do you conclude?
    1. Reject H₀: there is convincing evidence for Hₐ
    2. Accept H₀
    3. Hₐ is proven true

Summary card

Conclusions and errors

  • P ≤ α: reject H₀. Otherwise: fail to reject.
  • Type I: reject true H₀. Type II: keep false H₀.
  • Power rises with nn.

Watch out: never "accept" or "prove" H₀.

End of the lesson

Next lesson: The one-sample t test for a mean Back to the chapter

Answers: Conclusions, errors and power

A

  1. Reject H₀: there is convincing evidence for Hₐ

B

  1. Concluding the drug works when it really has no effect

C

  1. Use a larger sample

D

  1. Reject H₀: there is convincing evidence for Hₐ
  2. Reject H₀: there is convincing evidence for Hₐ
  3. Reject H₀: there is convincing evidence for Hₐ

CC BY-SA 4.0