9.4 Rare Events, the Sample, Decision and Conclusion

Using the Sample to Test the Null Hypothesis: p-Value

Note that hypothesis testing starts off with the assumption that the equality in the null hypothesis is true. Assuming the null hypothesis is true, the sampling distribution of the sample mean/proportion is used to find the probability of obtaining another sample as unusual as ours or more unusual than what we observed in the original sample. This probability is the p-value. It is represented on the sample statistic’s sampling distribution by the area to the tail from the sample statistic for one tail tests and twice the area to the tail from the sample statistic for two tail tests. For [latex]z[/latex] or [latex]t[/latex] distribution, we find this area using Desmos or StatKey. You can also use normalcdf and/or tcdf on your Ti calculator to find this area. If the test is two tailed, find the area to the tail from your sample statistic and double it to arrive at the p-value. Note that p-value doesn’t say anything about whether the null assumption is valid. The p­value is not the probability that the null hypothesis is true.

How to Correctly Interpret p-Values?  |   More on p-Value Interpretation

Level of significance [latex](\alpha)[/latex] is the probability of rejecting a null hypothesis when it is true (Type I Error). This level is set by the researcher. Significance level tells us the the probability that we will reject a null hypothesis that’s true. If we were to conduct another test using the same procedure and assumptions, then there is an [latex]\alpha%[/latex] chance of rejecting a true null hypothesis.

There are only two decisions we make:

1. Reject the Null Hypothesis → Sample data provides enough statistical evidence to support the alternative hypothesis. Test result is statistically significant.
2. Fail to reject the null hypothesis → Sample data does not provide enough statistical evidence to support the alternative hypothesis.
   Note: Fail to reject does not mean accept

p-Value based Decision

• If p-value [latex]< \alpha[/latex], ⇒ reject Ho ⇒ Test results are significant and sample data provides enough evidence to support Ha
• p-value [latex]\ge \alpha[/latex], ⇒ Fail to reject Ho ⇒ There’s not enough evidence to support Ha

Recap: What is a hypothesis test?

Statistical Significance and p-Values Explained Intuitively

Interpreting Hypothesis Test Results

So, you have stated the null and the alternative hypotheses, identified the claim, calculated the test statistic and the p-value (or identified the critical region), how do we interpret the results?

In hypothesis testing, we’re going to start off our analysis by assuming the NULL hypothesis is TRUE and at the end of our analysis, there are only two decisions we’re going to make:

• Reject the Null Hypothesis (which means that we were incorrect to assume the the null is true and so evidence we analyzed supports the alternative hypothesis)
• Fail to reject the Null Hypothesis (which means we couldn’t throw away our original assumption that the null is true and without throwing away the null hypothesis, we can’t support the alternative hypothesis)

You use either the p-value based decision method or the critical region method to either reject the null hypothesis or fail to reject the null hypothesis.

The next step is to determine the claim. Note that the claim could be either the null or the alternative hypothesis.

If the CLAIM is the Null Hypothesis:

• If decision was to Reject the null hypothesis, we reject the claim as the null is the claim.
  So, the interpretation is something like: “There is enough evidence to reject the claim that…”
• If the decision was to Fail to Reject the null hypothesis, we fail to reject the claim (since null is the claim) that..”

If the CLAIM is the Alternative Hypothesis:

• If the decision was to Reject the null hypothesis, we rejected the null so the alternative hypothesis must be assumed true for now until further evidence emerges. The interpretation is something like: “There is enough evidence to SUPPORT the claim that …” (Note the alternative hypothesis is the claim in this segment)
• If the decision was to Fail to Reject the null hypothesis, the sample didn’t provide the evidence to throw away the null assumption and the interpretation would be that “we don’t have evidence to support the claim (alternative hypothesis is the claim here) that …” This is so because the only way to support the alternative hypothesis is by rejecting the null hypothesis. If we can’t reject the null hypothesis, then we can’t say we can support the claim in the alternative hypothesis.

In summary: