View Lecture Slides with Transcript – Part C of Course Summary
Transcript – Unit 4B Case CQ Paired Samples A
Transcript – Unit 4B Case CQ Paired Samples C
This document is linked from Paired Samples
]]>This document linked from Case C→Q
]]>Related SAS Tutorials
Related SPSS Tutorials
We are in Case CQ of inference about relationships, where the explanatory variable is categorical and the response variable is quantitative.
As we mentioned in the summary of the introduction to Case C→Q, the first case that we will deal with is that involving matched pairs. In this case:
Notice from this point forward we will use the terms population 1 and population 2 instead of subpopulation 1 and subpopulation 2. Either terminology is correct.
One of the most common cases where dependent samples occur is when both samples have the same subjects and they are “paired by subject.” In other words, each subject is measured twice on the response variable, typically before and then after some kind of treatment/intervention in order to assess its effectiveness.
Suppose you want to assess the effectiveness of an SAT prep class.
It would make sense to use the matched pairs design and record each sampled student’s SAT score before and after the SAT prep classes are attended:
Recall that the two populations represent the two values of the explanatory variable. In this situation, those two values come from a single set of subjects.
This, however, is not the only case where the paired design is used. Other cases are when the pairs are “natural pairs,” such as siblings, twins, or couples.
Notes about graphical summaries for paired data in Case CQ:
The idea behind the paired ttest is to reduce this twosample situation, where we are comparing two means, to a single sample situation where we are doing inference on a single mean, and then use a simple ttest that we introduced in the previous module.
In this setting, we can easily reduce the raw data to a set of differences and conduct a onesample ttest.
In other words, by reducing the two samples to one sample of differences, we are essentially reducing the problem from a problem where we’re comparing two means (i.e., doing inference on μ_{1}−μ_{2}) to a problem in which we are studying one mean.
In general, in every matched pairs problem, our data consist of 2 samples which are organized in n pairs:
We reduce the two samples to only one by calculating the difference between the two observations for each pair.
For example, think of Sample 1 as “before” and Sample 2 as “after”. We can find the difference between the before and after results for each participant, which gives us only one sample, namely “before – after”. We label this difference as “d” in the illustration below.
The paired ttest is based on this one sample of n differences,
and it uses those differences as data for a onesample ttest on a single mean — the mean of the differences.
This is the general idea behind the paired ttest; it is nothing more than a regular onesample ttest for the mean of the differences!
We will now go through the 4step process of the paired ttest.
Recall that in the ttest for a single mean our null hypothesis was: Ho: μ = μ_{0} and the alternative was one of Ha: μ < μ_{0} or μ > μ_{0} or μ ≠ μ_{0}. Since the paired ttest is a special case of the onesample ttest, the hypotheses are the same except that:
Instead of simply μ we use the notation μ_{d} to denote that the parameter of interest is the mean of the differences.
In this course our null value μ_{0} is always 0. In other words, going back to our original paired samples our null hypothesis claims that that there is no difference between the two means. (Technically, it does not have to be zero if you are interested in a more specific difference – for example, you might be interested in showing that there is a reduction in blood pressure of more than 10 points but we will not specifically look at such situations).
Therefore, in the paired ttest: The null hypothesis is always:
Ho: μ_{d} = 0
(There IS NO association between the categorical explanatory variable and the quantitative response variable)
We will focus on the twosided alternative hypothesis of the form:
Ha: μ_{d} ≠ 0
(There IS AN association between the categorical explanatory variable and the quantitative response variable)
Some students find it helpful to know that it turns out that μ_{d} = μ_{1} – μ_{2} (in other words, the difference between the means is the same as the mean of the differences). You may find it easier to first think about the hypotheses in terms of μ_{1} – μ_{2} and then represent it in terms of μ_{d}.
The paired ttest, as a special case of a onesample ttest, can be safely used as long as:
The sample of differences is random (or at least can be considered random in context).
The distribution of the differences in the population should vary normally if you have small samples. If the sample size is large, it is safe to use the paired ttest regardless of whether the differences vary normally or not. This condition is satisfied in the three situations marked by a green check mark in the table below.
Note: normality is checked by looking at the histogram of differences, and as long as no clear violation of normality (such as extreme skewness and/or outliers) is apparent, the normality assumption is reasonable.
Assuming that we can safely use the paired ttest, the data are summarized by a test statistic:
where
This test statistic measures (in standard errors) how far our data are (represented by the sample mean of the differences) from the null hypothesis (represented by the null value, 0).
Notice this test statistic has the same general form as those discussed earlier:
As a special case of the onesample ttest, the null distribution of the paired ttest statistic is a t distribution (with n – 1 degrees of freedom), which is the distribution under which the pvalues are calculated. We will use software to find the pvalue for us.
As usual, we draw our conclusion based on the pvalue. Be sure to write your conclusions in context by specifying your current variables and/or precisely describing the population mean difference in terms of the current variables.
In particular, if a cutoff probability, α (significance level), is specified, we reject Ho if the pvalue is less than α. Otherwise, we fail to reject Ho.
If the pvalue is small, there is a statistically significant difference between what was observed in the sample and what was claimed in Ho, so we reject Ho.
Conclusion: There is enough evidence that the categorical explanatory variable is associated with the quantitative response variable. More specifically, there is enough evidence that the population mean difference is not equal to zero.
Remember: a small pvalue tells us that there is very little chance of getting data like those observed (or even more extreme) if the null hypothesis were true. Therefore, a small pvalue indicates that we should reject the null hypothesis.
If the pvalue is not small, we do not have enough statistical evidence to reject Ho.
Conclusion: There is NOT enough evidence that the categorical explanatory variable is associated with the quantitative response variable. More specifically, there is NOT enough evidence that the population mean difference is not equal to zero.
Notice how much better the first sentence sounds! It can get difficult to correctly phrase these conclusions in terms of the mean difference without confusing double negatives.
As in previous methods, we can followup with a confidence interval for the mean difference, μ_{d} and interpret this interval in the context of the problem.
Interpretation: We are 95% confident that the population mean difference (described in context) is between (lower bound) and (upper bound).
Confidence intervals can also be used to determine whether or not to reject the null hypothesis of the test based upon whether or not the null value of zero falls outside the interval or inside.
If the null value, 0, falls outside the confidence interval, Ho is rejected. (Zero is NOT a plausible value based upon the confidence interval)
If the null value, 0, falls inside the confidence interval, Ho is not rejected. (Zero IS a plausible value based upon the confidence interval)
NOTE: Be careful to choose the correct confidence interval about the population mean difference and not the individual confidence intervals for the means in the groups themselves.
Now let’s look at an example.
Note: In some of the videos presented in the course materials, we do conduct the onesided test for this data instead of the twosided test we conduct below. In Unit 4B we are going to restrict our attention to twosided tests supplemented by confidence intervals as needed to provide more information about the effect of interest.
Drunk driving is one of the main causes of car accidents. Interviews with drunk drivers who were involved in accidents and survived revealed that one of the main problems is that drivers do not realize that they are impaired, thinking “I only had 12 drinks … I am OK to drive.”
A sample of 20 drivers was chosen, and their reaction times in an obstacle course were measured before and after drinking two beers. The purpose of this study was to check whether drivers are impaired after drinking two beers. Here is a figure summarizing this study:
Since the measurements are paired, we can easily reduce the raw data to a set of differences and conduct a onesample ttest.
Here are some of the results for this data:
Step 1: State the hypotheses
We define μ_{d }= the population mean difference in reaction times (Before – After).
As we mentioned, the null hypothesis is:
The null hypothesis claims that the differences in reaction times are centered at (or around) 0, indicating that drinking two beers has no real impact on reaction times. In other words, drivers are not impaired after drinking two beers.
Although we really want to know whether their reaction times are longer after the two beers, we will still focus on conducting twosided hypothesis tests. We will be able to address whether the reaction times are longer after two beers when we look at the confidence interval.
Therefore, we will use the twosided alternative:
Step 2: Obtain data, check conditions, and summarize data
Let’s first check whether we can safely proceed with the paired ttest, by checking the two conditions.
We can see from the histogram above that there is no evidence of violation of the normality assumption (on the contrary, the histogram looks quite normal).
Also note that the vast majority of the differences are negative (i.e., the total reaction times for most of the drivers are larger after the two beers), suggesting that the data provide evidence against the null hypothesis.
The question (which the pvalue will answer) is whether these data provide strong enough evidence or not against the null hypothesis. We can safely proceed to calculate the test statistic (which in practice we leave to the software to calculate for us).
Test Statistic: We will use software to calculate the test statistic which is t = 2.58.
Step 3: Find the pvalue of the test by using the test statistic as follows
As a special case of the onesample ttest, the null distribution of the paired ttest statistic is a t distribution (with n – 1 degrees of freedom), which is the distribution under which the pvalues are calculated.
We will let the software find the pvalue for us, and in this case, gives us a pvalue of 0.0183 (SAS) or 0.018 (SPSS).
The small pvalue tells us that there is very little chance of getting data like those observed (or even more extreme) if the null hypothesis were true. More specifically, there is less than a 2% chance (0.018=1.8%) of obtaining a test statistic of 2.58 (or lower) or 2.58 (or higher), assuming that 2 beers have no impact on reaction times.
Step 4: Conclusion
In our example, the pvalue is 0.018, indicating that the data provide enough evidence to reject Ho.
Followup Confidence Interval:
As a followup to this conclusion, we quantify the effect that two beers have on the driver, using the 95% confidence interval for μ_{d}.
Using statistical software, we find that the 95% confidence interval for μ_{d}, the mean of the differences (before – after), is roughly (0.9, 0.1).
Note: Since the differences were calculated beforeafter, longer reaction times after the beers would translate into negative differences.
Since the confidence interval does not contain the null value of zero, we can use it to decide to reject the null hypothesis. Zero is not a plausible value of the population mean difference based upon the confidence interval. Notice that using this method is not always practical as often we still need to provide the pvalue in clinical research. (Note: this is NOT the interpretation of the confidence interval but a method of using the confidence interval to conduct a hypothesis test.)
Practical Significance:
We should definitely ask ourselves if this is practically significant and I would argue that it is.
In the output, we are generally provided the twosided pvalue. We must be very careful when converting this to a onesided pvalue (if this is not provided by the software)
The “driving after having 2 beers” example is a case in which observations are paired by subject. In other words, both samples have the same subject, so that each subject is measured twice. Typically, as in our example, one of the measurements occurs before a treatment/intervention (2 beers in our case), and the other measurement after the treatment/intervention.
Our next example is another typical type of study where the matched pairs design is used—it is a study involving twins.
Researchers have long been interested in the extent to which intelligence, as measured by IQ score, is affected by “nurture” as opposed to “nature”: that is, are people’s IQ scores mainly a result of their upbringing and environment, or are they mainly an inherited trait?
A study was designed to measure the effect of home environment on intelligence, or more specifically, the study was designed to address the question: “Are there statistically significant differences in IQ scores between people who were raised by their birth parents, and those who were raised by someone else?”
In order to be able to answer this question, the researchers needed to get two groups of subjects (one from the population of people who were raised by their birth parents, and one from the population of people who were raised by someone else) who are as similar as possible in all other respects. In particular, since genetic differences may also affect intelligence, the researchers wanted to control for this confounding factor.
We know from our discussion on study design (in the Producing Data unit of the course) that one way to (at least theoretically) control for all confounding factors is randomization—randomizing subjects to the different treatment groups. In this case, however, this is not possible. This is an observational study; you cannot randomize children to either be raised by their birth parents or to be raised by someone else. How else can we eliminate the genetics factor? We can conduct a “twin study.”
Because identical twins are genetically the same, a good design for obtaining information to answer this question would be to compare IQ scores for identical twins, one of whom is raised by birth parents and the other by someone else. Such a design (matched pairs) is an excellent way of making a comparison between individuals who only differ with respect to the explanatory variable of interest (upbringing) but are as alike as they can possibly be in all other important aspects (inborn intelligence). Identical twins raised apart were studied by Susan Farber, who published her studies in the book “Identical Twins Reared Apart” (1981, Basic Books).
In this problem, we are going to use the data that appear in Farber’s book in table E6, of the IQ scores of 32 pairs of identical twins who were reared apart.
Here is a figure that will help you understand this study:
Here are the important things to note in the figure:
Each of the 32 rows represents one pair of twins. Keeping the notation that we used above, twin 1 is the twin that was raised by his/her birth parents, and twin 2 is the twin that was raised by someone else. Let’s carry out the analysis.
Step 1: State the hypotheses
Recall that in matched pairs, we reduce the data from two samples to one sample of differences:
The hypotheses are stated in terms of the mean of the difference where, μ_{d} = population mean difference in IQ scores (Birth Parents – Someone Else):
Step 2: Obtain data, check conditions, and summarize data
Is it safe to use the paired ttest in this case?
The data don’t reveal anything that we should be worried about (like very extreme skewness or outliers), so we can safely proceed. Looking at the histogram, we note that most of the differences are negative, indicating that in most of the 32 pairs of twins, twin 2 (raised by someone else) has a higher IQ.
From this point we rely on statistical software, and find that:
Our test statistic is 1.85.
Our data (represented by the sample mean of the differences) are 1.85 standard errors below the null hypothesis (represented by the null value 0).
Step 3: Find the pvalue of the test by using the test statistic as follows
The pvalue is 0.074, indicating that there is a 7.4% chance of obtaining data like those observed (or even more extreme) assuming that H_{o} is true (i.e., assuming that there are no differences in IQ scores between people who were raised by their natural parents and those who weren’t).
Step 4: Conclusion
Using the conventional significance level (cutoff probability) of .05, our pvalue is not small enough, and we therefore cannot reject H_{o}.
Confidence Interval:
The 95% confidence interval for the population mean difference is (6.11322, 0.30072).
Interpretation:
This confidence interval does contain zero and thus results in the same conclusion to the hypothesis test. Zero IS a plausible value of the population mean difference and thus we cannot reject the null hypothesis.
Practical Significance:
It is very important to pay attention to whether the twosample ttest or the paired ttest is appropriate. In other words, being aware of the study design is extremely important. Consider our example, if we had not “caught” that this is a matched pairs design, and had analyzed the data as if the two samples were independent using the twosample ttest, we would have obtained a pvalue of 0.114.
Note that using this (wrong) method to analyze the data, and a significance level of 0.05, we would conclude that the data do not provide enough evidence for us to conclude that reaction times differed after drinking two beers. This is an example of how using the wrong statistical method can lead you to wrong conclusions, which in this context can have very serious implications.
Comments:
Now try a complete example for yourself.
Here are two other datasets with paired samples.
The statistical tests we have previously discussed (and many we will discuss) require assumptions about the distribution in the population or about the requirements to use a certain approximation as the sampling distribution. These methods are called parametric.
When these assumptions are not valid, alternative methods often exist to test similar hypotheses. Tests which require only minimal distributional assumptions, if any, are called nonparametric or distributionfree tests.
At the end of this section we will provide some details (see Details for NonParametric Alternatives), for now we simply want to mention that there are two common nonparametric alternatives to the paired ttest. They are:
The fact that both of these tests have the word “sign” in them is not a coincidence – it is due to the fact that we will be interested in whether the differences have a positive sign or a negative sign – and the fact that this word appears in both of these tests can help you to remember that they correspond to paired methods where we are often interested in whether there was an increase (positive sign) or a decrease (negative sign).
Recall the roletype classification table framing our discussion on inference about the relationship between two variables.
We start with case C→Q, where the explanatory variable is categorical and the response variable is quantitative.
Recall that in the Exploratory Data Analysis unit, examining the relationship between X and Y in this situation amounts, in practice, to:
To do that, we used
We will need to add one layer of difficulty here with the possibility that we may have paired or matched samples as opposed to independent samples or groups. Note that all of the examples we discussed in Case CQ in Unit 1 consisted of independent samples.
First we will review the general scenario.
To understand the logic, we’ll start with an example and then generalize.
Suppose that our variable of interest is the GPA of college students in the United States. From Unint 4A, we know that since GPA is quantitative, we will conduct inference on μ, the (population) mean GPA among all U.S. college students.
Since this section is about relationships, let’s assume that what we are really interested in is not simply GPA, but the relationship between:
In other words, we want to explore whether GPA is related to year in college.
The way to think about this is that the population of U.S. college students is now broken into 4 subpopulations: freshmen, sophomores, juniors and seniors. Within each of these four groups, we are interested in the GPA.
The inference must therefore involve the 4 subpopulation means:
It makes sense that the inference about the relationship between year and GPA has to be based on some kind of comparison of these four means.
If we infer that these four means are not all equal (i.e., that there are some differences in GPA across years in college) then that’s equivalent to saying GPA is related to year in college. Let’s summarize this example with a figure:
In general, making inferences about the relationship between X and Y in Case C→Q boils down to comparing the means of Y in the subpopulations, which are created by the categories defined by X (say k categories). The following figure summarizes this:
We will split this into two different scenarios (k = 2 and k > 2), where k is the number of categories defined by X.
For example:
Furthermore, within the scenario of comparing two means (i.e., examining the relationship between X and Y, when X has only two categories, k = 2) we will distinguish between two scenarios.
Here, the distinction is somewhat subtle, and has to do with how the samples from each of the two subpopulations we’re comparing are chosen. In other words, it depends upon what type of study design will be implemented.
We have learned that many experiments, as well as observational studies, make a comparison between two groups (subpopulations) defined by the categories of the explanatory variable (X), in order to see if the response (Y) differs.
In some situations, one group (subpopulation 1) is defined by one category of X, and another independent group (subpopulation 2) is defined by the other category of X. Independent samples are then taken from each group for comparison.
Suppose we are conducting a clinical trial. Participants are randomized into two independent subpopulations:
Each individual appears in only one of these two groups and individuals are not matched or paired in any way. Thus the two samples or groups are independent. We can say those given the drug are independent from those given the placebo.
Recall: By randomly assigning individuals to the treatment we control for both known and unknown lurking variables.
Suppose the Highway Patrol wants to study the reaction times of drivers with a blood alcohol content of half the legal limit in their state.
An observational study was designed which would also serve as publicity on the topic of drinking and driving. At a large event where enough alcohol would be consumed to obtain plenty of potential study participants, officers set up an obstacle course and provided the vehicles. (Other considerations were also implemented to keep the car and track conditions consistent for each participant.)
Volunteers were recruited from those in attendance and given a breathalyzer test to determine their blood alcohol content. Two types of volunteers were chosen to participate:
Here also, we have two independent groups – even if originally they were taken from the same sample of volunteers – each individual appears in only one of the two groups, the comparison of the reaction times is a comparison between two independent groups.
However, in this study, there was NO random assignment to the treatment and so we would need to be much more concerned about the possibility of lurking variables in this study compared to one in which individuals were randomized into one of these two groups.
We will see it may be more appropriate in some studies to use the same individual as a subject in BOTH treatments – this will result in dependent samples.
When a matched pairs sample design is used, each observation in one sample is matched/paired/linked with an observation in the other sample. These are sometimes called “dependent samples.”
Matching could be by person (if the same person is measured twice), or could actually be a pair of individuals who belong together in a relevant way (husband and wife, siblings).
In this design, then, the same individual or a matched pair of individuals is used to make two measurements of the response – one for each of the two levels of the categorical explanatory variable.
Advantages of a paired sample approach include:
Disadvantages of a paired sample approach include:
Suppose we are conducting a study on a pain blocker which can be applied to the skin and are comparing two different dosage levels of the solution which in this study will be applied to the forearm.
For each participant both solutions are applied with the following protocol:
Here we have dependent samples since the same patient appears in both dosage groups.
Again, randomization is employed to help minimize other issues related to study design such as an order or testing effect.
Suppose the department of motor vehicles wants to check whether drivers are impaired after drinking two beers.
The reaction times (measured in seconds) in an obstacle course are measured for 8 randomly selected drivers before and then after the consumption of two beers.
We have a matchedpairs design, since each individual was measured twice, once before and once after.
In matched pairs, the comparison between the reaction times is done for each individual.
Comment:
We will begin our discussion of Inference for Relationships with Case CQ, where the explanatory variable (X) is categorical and the response variable (Y) is quantitative. We discussed that inference in this case amounts to comparing population means.
Now test your skills at identifying the three scenarios in Case CQ.
Here is a summary of the tests we will learn for the scenario where k = 2.
Independent Samples (More Emphasis) 
Dependent Samples (Less Emphasis) 
Standard Tests
NonParametric Test

Standard Test
NonParametric Tests

Here is a summary of the tests we will learn for the scenario where k > 2.
Independent Samples (Only Emphasis) 
Dependent Samples (Not Discussed) 
Standard Tests
NonParametric Test

Standard Test
