Wrap-Up (Case C-Q)
We are now done with case C→Q.
- We learned that this case is further classified into sub-cases, depending on the number of groups that we are comparing (i.e., the number of categories that the explanatory variable has), and the design of the study (independent vs. dependent samples).
- For each of the three sub-cases that we covered, we learned the appropriate inferential method, and emphasized the idea behind the method, the conditions under which it can be safely used, how to carry it out using software, and the interpretation of the results.
- We also learned which non-parametric tests are applicable and under what circumstances they might be used instead of the standard methods.
The following table summarizes when each of the three standard tests, covered in this module, are used:
The following summary discusses each of the above named sub-cases of C→Q within the context of the hypothesis testing process.
Step 1: Stating the null and alternative hypotheses (H0 and Ha)
- Although the one-sided alternatives are provided here where possible, remember that we will focus only on two-sided tests supplemented by confidence intervals for methods in Unit 4B.
Step 2: Check Conditions and Summarize the Data Using a Test Statistic
We need to check that the conditions under which the test can be reliably used are met.
For the Paired t-test (as a special case of a one-sample t-test), the conditions are:
- The sample of differences is random (or at least can be considered so in context).
- We are in one of the three situations marked with a green check mark in the following table:
For the Two-Sample t-test, the conditions are:
- Two samples are independent and random
- One of the following two scenarios holds:
- Both populations are normal
- Populations are not normal, but large sample size (>30)
For an ANOVA, the conditions are:
- The samples drawn from each of the populations being compared are independent.
- The response variable varies normally within each of the populations being compared. As is often the case, we do not have to worry about this assumption for large sample sizes.
- The populations all have the same standard deviation.
Now we summarize the data using a test statistic.
- Although we will not be calculating these test statistics by hand, we will review the formulas for each test statistic here.
For the Paired t-test the test statistic is:
For the Two-Sample t-test assuming equal variances the test statistic is:
For the Two-Sample t-test assuming unequal variances the test statistic is:
For an ANOVA the test statistic is:
Step 3: Finding the p-value of the test
Use statistical software to determine the p-value.
- The p-value is the probability of getting data like those observed (or even more extreme) assuming that the null hypothesis is true, and is calculated using the null distribution of the test statistic.
- The p-value is a measure of the evidence against H0.
- The smaller the p-value, the more evidence the data present against H0.
The p-values for three C→Q tests are obtained from the output.
Step 4: Making conclusions
Conclusions about the significance of the results:
- If the p-value is small, the data present enough evidence to reject Ho (and accept Ha).
- If the p-value is not small, the data do not provide enough evidence to reject H0.
- To help guide our decision, we use the significance level as a cutoff for what is considered a small p-value. The significance cutoff is usually set at .05, but should not be considered inviolable.
Conclusions should always be stated in the context of the problem and can all be written in the basic form below:
- There (IS or IS NOT) enough evidence that there is an association between (X) and (Y). Where X and Y should be given in context.
Following the test…
- For a paired t-test, a 95% confidence interval for μd can be very insightful after a test has rejected the null hypothesis, and can also be used for testing in the two-sided case.
- For a two-sample t-test, a 95% confidence interval for μ1−μ2 can be very insightful after a test has rejected the null hypothesis, and can also be used for testing in the two-sided case.
- If the ANOVA F-test has rejected the null hypothesis, looking at the confidence intervals for the population means that are in the output can provide visual insight into why the H0 was rejected (i.e., which of the means differ).
- For a Paired t-test we might investigate using the Wilcoxon Signed-Rank test or the Sign test.
- For a Two-Sample t-test we might investigate using the Wilcoxon Rank-Sum test (Mann-Whitney U test).
- For an ANOVA we might investigate using the Kruskal-Wallis test.