Decision Errors

Schwab

Hypothesis Testing with p-values

  1. Write notation

  2. Check Conditions

  3. Build Theoretical Distributions with assumed parameter.

  4. Find the probability of a sample statistic.

The question we are trying to answer:

If the null hypothesis is true, how likely is it that we would have gotten the statistic from our sample?

Example from difference of props

If \(H_0 : p_1 = p_2\) how likely is it we would have gotten \(\widehat{p}_1- \widehat{p}_2 = 0.13\)

With a p-value of 0.086 found it to be likely that we would have gotten 0.13.

Note on alpha \(\alpha\) :

\(\alpha\) is a cut off value. If the p-value is larger than \(\alpha\) we say \(\widehat{p}\) is likely (Fail to reject \(H_0\)), smaller than alpha \(\widehat{p}\) is unlikely (reject \(H_0\)).

  • \(\alpha\) shows the strength of our evidence.

  • weak evidence \(\alpha = 0.1\) or higher.

  • moderate evidence \(\alpha =0.05\)

  • strong evidence \(\alpha = 0.01\)

Choosing alpha involves making trade offs that are beyond the scope of this class.

Making a wrong decision

These are just probabilities. They could be wrong.

  • We could reject \(H_0\) if it is true. (Type 1)

or

  • We could fail to reject \(H_0\) if it is false. (Type 2)

Decision Errors