9.2 Outcomes and the Type I and Type II Errors

In hypothesis testing, we start off by assuming the null hypothesis is true. That is, we assume the population parameter is the one listed in the null hypothesis and use that to perform calculations. At the end of our analysis, there are only two possible decisions we will make:

1. Reject the Null Hypothesis → Sample data provides enough statistical evidence to support the alternative hypothesis. Test result is statistically significant.        OR
2. Fail to reject the null hypothesis → Sample data does not provide enough statistical evidence to support the alternative hypothesis.
   Note: Fail to reject does not mean accept

Types of Errors: Type I and Type II
Since we generally do not know the population parameter, we do not know if the null hypothesis that we assumed to be true is, in fact, true in reality or not. That means, we do not know whether the decision we make is the correct one or not. If the decision is to reject the null hypothesis, then it would be a good decision if, in reality, the null assumption was incorrect and a bad decision (Type I Error) if the null assumption was correct. Similarly, a decision to fail to reject the null would be good if the null assumption is true in reality and bad otherwise. All in all, we  have a total of four possible outcomes when performing a hypothesis test. Of those outcomes, Type I error occurs when we reject the null hypothesis when, in reality, it is true. Note that if we reject the null hypothesis, our conclusion would be to support the alternative. Type II error is when we fail to reject a false null hypothesis. Probability of making type II error = [latex]\beta[/latex] (beta). The power of a test is [latex]1 - \beta[/latex]. Ideally, we want a high power that is as close to one as possible. Increasing the sample size can increase the power of the Test.

Level of significance [latex](\alpha)[/latex] is the probability of rejecting a null hypothesis when it is true (the probability of making a Type I Error). This level is set by the researcher. Significance level tells us the the probability that we will reject a null hypothesis that’s true. If we were to conduct another test using the same procedure and assumptions, then there is an [latex]\alpha%[/latex] chance of rejecting a true null hypothesis.