Details for Non-Parametric Alternatives in Case C-Q
Related SAS Tutorials
- 7E – (4:34) Non Parametric Tests for independent samples (k= 2 and k > 2)
- 8C – (5:20) Paired T-Test and Non Parametric Tests for dependent samples
Related SPSS Tutorials
- 7E – (3:57) Non Parametric Tests for independent samples (k= 2 and k > 2)
- 8D – (3:32) Non Parametric (Paired) for dependent samples
- Non-Parametric Tests
- Case C-Q – Matched Pairs
- Case C-Q – Two Independent Samples – Wilcoxon Rank-Sum Test (Mann-Whitney U)
- Case CQ – K > 2 – The Kruskal-Wallis Test
- Let’s Summarize
We mentioned some non-parametric alternatives to the paired t-test, two-sample t-test for independent samples, and the one-way ANOVA.
Here we provide more details and resources for these tests for those of you who wish to conduct them in practice.
Non-Parametric Tests
The statistical tests we have previously discussed require assumptions about the distribution in the population or about the requirements to use a certain approximation as the sampling distribution. These methods are called parametric.
When these assumptions are not valid, alternative methods often exist to test similar hypotheses. Tests which require only minimal distributional assumptions, if any, are called non-parametric or distribution-free tests.
In some cases, these tests may be called exact tests due to the fact that their methods of calculating p-values or confidence intervals require no mathematical approximation (a foundation of many statistical methods).
However, note that when the assumptions are precisely satisfied, some “parametric” tests can also be considered “exact.”
Case CQ – Matched Pairs
We will look at two non-parametric tests in the paired sample setting.
The Sign Test
The sign test is a very general test used to compare paired samples. It can be used instead of the Paired T-test if the assumptions are not met although the next test we discuss is likely a better option in that case as we will see. However, the sign test does have some advantages and is worth understanding.
- The idea behind the test is to find the sign of the differences (positive or negative) and use this information to determine if the medians between the two groups are the same.
- If the two paired measurements came from the populations with equal medians, we would expect half of the differences to be positive and half to be negative. Thus the sampling distribution of our statistic is simply a binomial with p = 0.5.
Outline of Procedure for the SIGN TEST
- Step 1: State the hypotheses
The hypotheses are:
Ho: the medians are equal
Ha: the medians are not equal (one-sided tests are possible)
- Step 2: Obtain data, check conditions, and summarize data
We require a random sample (or at least can be considered random in context).
The sign test can be used for any data for which the sign of the difference can be obtained. Thus, it can be used for:
quantitative measures (continuous or discrete)
Examples: Systolic Blood Pressure, Number of Drinks
(categorical) ordinal measures
Examples: Rating scales, Letter Grades
(categorical) binary measures where we can only tell whether one pair is “larger” or “smaller” compared to the other pair
Examples: Is the left arm more or less sunburned than the right arm?, Was there an improvement in pain after treatment?
For this reason, this test is very widely applicable!
The data are summarized by a test statistic which counts the number of positive (or negative) differences. Any ties (zero differences) are discarded.
- Step 3: Find the p-value of the test by using the test statistic as follows
The p-values are calculated using the binomial distribution (or a normal approximation for large samples). We will rely on software to obtain the p-value for this test.
- Step 4: Conclusion
The decision is made in the same manner as other tests.
We can word our conclusion in terms of the medians in the two populations or in terms of the relationship between the categorical explanatory variable (X) and the response variable (Y).
The Wilcoxon Signed-Rank Test:
The Wilcoxon signed-rank Test is a general test to compare distributions in paired samples. This test is usually the preferred alternative to the Paired t-test when the assumptions are not satisfied.
The idea behind the test is to determine if the two populations seem to be the same or different based upon the ranks of the absolute differences (instead of the magnitude of the differences). Ranking procedures are commonly used in non-parametric methods as this moderates the effect of any outliers.
We have one assumption for this test. We assume the distribution of the differences is symmetric.
Under this assumption, if the two paired measurements came from the populations with equal means/medians, we would expect the two sets of ranks (those for positive differences and those for negative differences) to be distributed similarly. If there is a large difference here, this gives evidence of a true difference.
Outline of Procedure for the Wilcoxon Signed-Rank Test
- Step 1: State the hypotheses
The hypotheses are:
Ho: the means/medians are equal
Ha: the means/medians are not equal (one-sided tests are possible)
- Step 2: Obtain data, check conditions, and summarize data
We have a random sample and we assume the distribution of the differences is symmetric so we should check to be sure that there is no clear skewness to the distribution of the differences.
The Wilcoxon signed-Rank test can be used for quantitative or ordinal data (but not binary as for the sign test).
The data are summarized by a test statistic which counts the sum of the positive (or negative) ranks. Any zero differences are discarded.
To rank the pairs, we find the differences (much as we did in the paired t-test), take the absolute value of these differences and rank the pairs from 1 = smallest non-zero difference to m = largest non-zero difference, where m = number of non-zero pairs.
Then we determine which ranks came from positive (or negative) differences and find the sum of these ranks.
You will not be conducting this test by hand. We simply wish to explain some of the logic behind the scenes for these tests.
- Step 3: Find the p-value of the test by using the test statistic as follows
The p-values are calculated using a distribution specific to this test. We will rely on software to obtain the p-value for this test.
- Step 4: Conclusion
The decision is made in the same manner as other tests. We can word our conclusion in terms of the means or medians in the two populations or in terms of the existence or non-existence of a relationship between the categorical explanatory variable (X) and the response variable (Y).
Comments:
- The sign test tends to have much lower power than the paired t-test or the Wilcoxon signed-Rank test. In other words, the sign test has less chance of being able to detect a true difference than the other tests. It is, however, applicable in the case where we only know “better” or “worse” for each pair, where the other two methods are not.
- The Wilcoxon signed-rank test is comparable to the paired t-test in power and can even perform better than the paired t-test under certain conditions. In particular, this can occur when there are a few very large outliers as these outliers can greatly affect our estimate of the standard error in the paired t-test since it is based upon the sample standard deviation which is highly affected by such outliers.
- Both the sign Test and the Wilcoxon signed-rank test can also be used for one sample. In that case, you must specify the null value and calculate differences between the observed value and the null value (instead of the difference between two pairs).
Case CQ – Two Independent Samples – Wilcoxon Rank-Sum Test (Mann-Whitney U):
We will look at one non-parametric test in the two-independent samples setting.
The Wilcoxon rank-sum test (Mann-Whitney U test) is a general test to compare two distributions in independent samples. It is a commonly used alternative to the two-sample t-test when the assumptions are not met.
The idea behind the test is to determine if the two populations seem to be the same or different based upon the ranks of the values instead of the magnitude. Ranking procedures are commonly used in non-parametric methods as this moderates the effect of any outliers.
There are many ways to formulate this test. For our purposes, we will assume the quantitative variable (Y) is a continuous random variable (or can be treated as continuous, such as for very large counts) and that we are interested in testing whether there is a “shift” in the distribution. In other words, we assume that the distribution is the same except that in one group the distribution is higher (or lower) than in the other.
- Step 1: State the hypotheses
We assume the distributions of the two populations are the same except for a horizontal shift in location.
The hypotheses are:
Ho: the medians are equal
Ha: the medians are not equal (one-sided tests are possible)
- Step 2: Obtain data, check conditions, and summarize data
(i) We have two independent random samples. All observations in each sample must be independent of all other observations.
(ii) The version of the Wilcoxon rank-sum test (Mann-Whitney U test) we are using assumes a that the quantitative response variable is a continuous random variable.
(iii) We assume there is only a location shift so we should check that the two distributions are similar except possibly for their locations.
(iv) The data are summarized by a test statistic which counts the sum of the sample 1 (or sample 2) ranks.
To rank the observations, we combine all observations in both samples and rank from smallest to largest.
Then we determine which ranks came from sample 1 (or sample 2) and find the sum of these ranks.
You will not be conducting this test by hand. We simply wish to explain some of the logic behind the scenes for these tests.
- Step 3: Find the p-value of the test by using the test statistic as follows
The p-values are calculated using a distribution specific to this test. We will rely on software to obtain the p-value for this test.
- Step 4: Conclusion
The decision is made in the same manner as other tests. We can word our conclusion in terms of the medians in the two populations or in terms of the existence or non-existence of a relationship between the categorical explanatory variable (X) and the response variable (Y).
Case CQ – K > 2 – The Kruskal-Wallis Test
We will look at one non-parametric test in the k > 2 independent sample setting.
The Kruskal-Wallis test is a general test to compare multiple distributions in independent samples.
The idea behind the test is to determine if the k populations seem to be the same or different based upon the ranks of the values instead of the magnitude. Ranking procedures are commonly used in non-parametric methods as this moderates the effect of any outliers.
The test assumes identically-shaped and scaled distributions for each group, except for any difference in medians.
Step 1: State the hypothesesThe hypotheses are:
- Ho: the medians of all groups are equal
- Ha: the medians are not all equal
Step 2: Obtain data, check conditions, and summarize data
(i) We have independent random samples from our k populations. All observations in each sample must be independent of all other observations.
(ii) We have an ordinal, discrete, or continuous response variable Y.
(iii) We assume there is only a location shift so we should check that the distributions are similar except possibly for their locations.
(iv) The data are summarized by a test statistic which involves the ranks of observations in each group.
To rank the observations, we combine all observations in all samples and rank from smallest to largest.
Then we determine which ranks came from which sample and use these to obtain the test statistic.
Step 3: Find the p-value of the test by using the test statistic as follows
The p-values are calculated using a distribution specific to this test. We will rely on software to obtain the p-value for this test.
Step 4: Conclusion
The decision is made in the same manner as other tests. We can word our conclusion in terms of the medians in the k populations or in terms of the existence or non-existence of a relationship between the categorical explanatory variable (X) and the response variable (Y).
Let’s Summarize
- We have presented the basic idea for the non-parameteric alternatives for Case C-Q
- The sign test and the Wilcoxon signed-rank test are possible alternatives to the paired t-test in the case of two dependent samples.
- The Wilcoxon rank-sum test (also known as the Mann-Whitney U test) is a possible alternative to the two-sample t-test in the case of two independent samples.
- The Kruskal-Wallis test is a possible alternative to the one-way ANOVA in the case of more than two independent samples.
- In this course, we simply want you to be aware of which non-parameteric alternatives are commonly used to address issues with the assumptions.
- We are not asking you to conduct these tests but we do still provide information for those interested in being able to conduct these tests in practice.