Introduction to Probability
- Probability is Not Always Intuitive
- Need a Laugh?
- What is Probability?
- Probability is the Likelihood of Something Happening
- Determining Probability
- To Summarize So Far
- Relative Frequency
- Let’s Summarize
Now that we understand how probability fits into the Big Picture as a key element behind statistical inference, we are ready to learn more about it. Our first goal is to introduce some fundamental terminology (the language) and notation that is used when discussing probability.
Although most of the probability calculations we will conduct will be rather intuitive due to their simplicity, we start with two fun examples that will illustrate the interesting and sometimes complex nature of probability.
Often, relying only on our intuition is not enough to determine probability, so we’ll need some tools to work with, which is exactly what we’ll study in this section.
Here is the first of two motivating examples:
If this example still did not persuade you that probability is not always intuitive, the next example should definitely do the trick.
From these two examples, (maybe) you have seen that your original hunches cannot always be counted upon to give you correct predictions of probabilities.
We won’t think any more about these examples as they are from the “harder” end of the complexity spectrum but hopefully they have motivated you to learn more about probability and you do not need to be convinced of their solution to continue!
In general, probability is not always intuitive.
Watch this (funny) video which has an excellent point about “how probability DOES NOT work”: clip from the Daily Show with Jon Stewart about the Large Hadron Collider (5:58).
It is possible viewers in other countries may not be able to view the clip from this source. You may or may not be able to find it online through searching. Here is the transcript summary I sometimes use in class to get the point across (it isn’t quite as funny but I think you can still figure out what is wrong here):
- John Oliver: So, roughly speaking, what are the chances that the world is going to be destroyed? (by the large hadron collider) One-in-a-million? One-in-a-billion?
- Walter: Well, the best we can say right now is about a one-in-two chance.
- John Oliver: 50-50?
- Walter: Yeah, 50-50… It’s a chance; it’s a 50-50 chance.
- John Oliver: You keep coming back to this 50-50 thing, it’s weird Walter.
- Walter: Well, if you have something that can happen and something that won’t necessarily happen, it’s going to either happen or it’s going to not happen. And, so, it’s … the best guess is 1 in 2.
- John Oliver: I’m not sure that’s how probability works, Walter.
And … John Oliver is correct! :-)
Eventually we will need to develop a more formal approach to probability, but we will begin with an informal discussion of what probability is.
Probability is a mathematical description of randomness and uncertainty. It is a way to measure or quantify uncertainty. Another way to think about probability is that it is the official name for “chance.”
One way to think of probability is that it is the likelihood that something will occur.
Probability is used to answer the following types of questions:
- What is the chance that it will rain tomorrow?
- What is the chance that a stock will go up in price?
- What is the chance that I will have a heart attack?
- What is the chance that I will live longer than 70 years?
- What is the likelihood that when rolling a pair of dice, I will roll doubles?
- What is the probability that I will win the lottery?
- What is the probability that I will become diabetic?
Each of these examples has some uncertainty. For some, the chances are quite good, so the probability would be quite high. For others, the chances are not very good, so the probability is quite low (especially winning the lottery).
Certainly, the chance of rain is different each day, and is higher during some seasons. Your chance of having a heart attack, or of living longer than 70 years, depends on things like your current age, your family history, and your lifestyle. However, you could use your intuition to predict some of those probabilities fairly accurately, while others you might have no instinct about at all.
We think you will agree that the word probability is a bit long to include in equations, graphs and charts, so it is customary to use some simplified notation instead of the entire word.
If we wish to indicate “the probability it will rain tomorrow,” we use the notation “P(rain tomorrow).” We can abbreviate the probability of anything. If we let A represent what we wish to find the probability of, then P(A) would represent that probability.
We can think of “A” as an “event.”
|P(win lottery)||the probability that a person who has a lottery ticket will win that lottery|
|P(A)||the probability that event A will occur|
|P(B)||the probability that event B will occur|
What values can the probability of an event take, and what does the value tell us about the likelihood of the event occurring?
Many people prefer to express probability in percentages. Since all probabilities are decimals, each can be changed to an equivalent percentage. Thus, the latest principle is equivalent to saying, “The chance that an event will occur is between 0% and 100%.”
Probabilities can be determined in two fundamental ways. Keep reading to find out what they are.
There are 2 fundamental ways in which we can determine probability:
- Theoretical (also known as Classical)
- Empirical (also known as Observational)
Classical methods are used for games of chance, such as flipping coins, rolling dice, spinning spinners, roulette wheels, or lotteries.
The probabilities in this case are determined by the game (or scenario) itself and are often found relatively easily using logic and/or probability rules.
Although we will not focus on this type of probability in this course, we will mention a few examples to get you thinking about probability and how it works.
We will now shift our discussion to empirical ways to determine probabilities.
A single flip of a coin has an uncertain outcome. So, every time a coin is flipped, the outcome of that flip is unknown until the flip occurs.
However, if you flip a fair coin over and over again, would you expect P(H) to be exactly 0.5? In other words, would you expect there to be the same number of results of “heads” as there are “tails”?
The following activity will allow you to discover the answer.
The above Learn by Doing activity was our first example of the second way of determining probability: Empirical (Observational) methods. In the activity, we determined that the probability of getting the result “heads” is 0.5 by flipping a fair coin many, many times.
A Second Question
After doing this experiment, an important question naturally comes to mind. How would we know if the coin was not fair? Certainly, classical probability methods would never be able to answer this question. In addition, classical methods could never tell us the actual P(H). The only way to answer this question is to perform another experiment.
The next activity will allow you to do just that.
So, these types of experiments can verify classical probabilities and they can also determine when games of chance are not following fair practices. However, their real importance is to answer probability questions that arise when we are faced with a situation that does not follow any pattern and cannot be predetermined. In reality, most of the probabilities of interest to us fit the latter description.
- Probability is a way of quantifying uncertainty.
- We are interested in the probability of an event — the likelihood of the event occurring.
- The probability of an event ranges from 0 to 1. The closer the probability is to 0, the less likely the event is to occur. The closer the probability is to 1, the more likely the event is to occur.
- There are two ways to determine probability: Theoretical (Classical) and Empirical (Observational).
- Theoretical methods use the nature of the situation to determine probabilities.
- Empirical methods use a series of trials that produce outcomes that cannot be predicted in advance (hence the uncertainty).
If we toss a coin, roll a die, or spin a spinner many times, we hardly ever achieve the exact theoretical probabilities that we know we should get, but we can get pretty close. When we run a simulation or when we use a random sample and record the results, we are using empirical probability. This is often called the Relative Frequency definition of probability.
Here is a realistic example where the relative frequency method was used to find the probabilities:
Let’s review the relative frequency method for finding probabilities:
To estimate the probability of event A, written P(A), we may repeat the random experiment many times and count the number of times event A occurs. Then P(A) is estimated by the ratio of the number of times A occurs to the number of repetitions, which is called the relative frequency of event A.
So, we’ve seen how the relative frequency idea works, and hopefully the activities have convinced you that the relative frequency of an event does indeed approach the theoretical probability of that event as the number of repetitions increases. This is called the Law of Large Numbers.
The Law of Large Numbers states that as the number of trials increases, the relative frequency becomes the actual probability. So, using this law, as the number of trials increases, the empirical probability gets closer and closer to the theoretical probability.
- Note that the relative frequency approach provides only an estimate of the probability of an event. However, we can control how good this estimate is by the number of times we repeat the random experiment. The more repetitions that are performed, the closer the relative frequency gets to the true probability of the event.
- One interesting question would be: “How many times do I need to repeat the random experiment in order for the relative frequency to be, say, within 0.001 of the actual probability of the event?” We will come back to that question in the inference section.
- A pedagogical comment: We’ve introduced relative frequency here in a more practical approach, as a method for estimating the probability of an event. More traditionally, relative frequency is not presented as a method, but as a definition:
- There are many situations of interest in which physical circumstances do not make the probability obvious. In fact, most of the time it is impossible to find the theoretical probability, and we must use empirical probabilities instead.
Probability is a way of quantifying uncertainty. In this section, we defined probability as the likelihood or chance that something will occur and introduced the basic notation of probability such as P(win lottery).
You have seen that all probabilities are values between 0 and 1, where an event with no chance of occurring has a probability of 0 and an event which will always occur has a probability of 1.
We have discussed the two primary methods of calculating probabilities
- Theoretical or Classical Probability: uses the nature of the situation to determine probabilities
- Empirical or Observational Probability: uses a series of trials that produce outcomes that cannot be predicted in advance (hence the uncertainty)
In our course we will focus on Empirical probability and will often calculate probabilities from a sample using relative frequencies.
This is useful in practice since the Law of Large Numbers allows us to estimate the actual (or true) probability of an event by the relative frequency with which the event occurs in a long series of trials. We can collect this information as data and we can analyze this data using statistics.