Comprehensive Guide to Probability
Table of Contents
Probability Basics
Probability is a measure of the likelihood that an event will occur. It is quantified as a number between 0 and 1, where 0 indicates impossibility and 1 indicates certainty.
Basic Probability Formula
The probability of an event A is given by:
P(A) = Number of favorable outcomes / Total number of possible outcomes
Key Terms
- Experiment: Any procedure that can be repeated and has a well-defined set of possible outcomes.
- Sample Space (S): The set of all possible outcomes of an experiment.
- Event: A subset of the sample space, or a set of possible outcomes.
- Outcome: A single result of an experiment.
- Probability: A measure of the likelihood of an event occurring.
Example: Coin Toss
When tossing a fair coin once:
- Sample Space S = {Heads, Tails}
- For event A = "getting Heads": P(A) = 1/2 = 0.5 or 50%
- For event B = "getting Tails": P(B) = 1/2 = 0.5 or 50%
Types of Probability
Classical (Theoretical) Probability
Based on equally likely outcomes and determined by logical analysis rather than observation or personal belief.
Example: The probability of rolling a 3 on a fair six-sided die.
P(rolling a 3) = 1/6 ≈ 0.167
Explanation: There are 6 equally likely outcomes (1, 2, 3, 4, 5, 6), and only one of them is a "3".
Empirical (Experimental or Relative Frequency) Probability
Based on observations and actual experiments, calculated as the relative frequency of an event.
P(A) = Number of times event A occurred / Total number of trials
Example: If a coin is tossed 100 times and heads appears 53 times, the empirical probability of heads is:
P(Heads) = 53/100 = 0.53 or 53%
Subjective Probability
Based on personal judgment, experience, or belief about the likelihood of an event occurring.
Example: A weather forecaster might assign a 70% chance of rain tomorrow based on their analysis of weather patterns and experience.
Axiomatic Probability
Based on a set of axioms or rules that probability must satisfy. This approach was developed by Kolmogorov and forms the mathematical foundation of probability theory.
Kolmogorov's Axioms:
- The probability of any event is a non-negative real number: P(A) ≥ 0
- The probability of the entire sample space is 1: P(S) = 1
- For mutually exclusive events, the probability of their union equals the sum of their individual probabilities: P(A ∪ B) = P(A) + P(B)
Conditional Probability
The probability of an event occurring given that another event has already occurred.
P(A|B) = P(A ∩ B) / P(B)
where P(A|B) reads as "the probability of A given B"
Example: In a deck of 52 cards, what is the probability of drawing a king given that you have drawn a face card?
Face cards include kings, queens, and jacks. Total face cards = 12.
Number of kings = 4
P(King|Face card) = P(King ∩ Face card) / P(Face card) = 4/12 = 1/3
Probability Laws and Formulas
Addition Law
For any two events A and B:
P(A or B) = P(A ∪ B) = P(A) + P(B) - P(A ∩ B)
If A and B are mutually exclusive (cannot occur together), then:
P(A ∪ B) = P(A) + P(B)
Example: Addition Law
In a standard deck of 52 cards, what is the probability of drawing either a heart or a face card?
Let H = "drawing a heart" and F = "drawing a face card"
P(H) = 13/52 = 1/4 (There are 13 hearts in a deck)
P(F) = 12/52 = 3/13 (There are 12 face cards: jacks, queens, and kings)
P(H ∩ F) = 3/52 (There are 3 face cards that are hearts)
Using the addition rule:
P(H ∪ F) = P(H) + P(F) - P(H ∩ F)
P(H ∪ F) = 1/4 + 3/13 - 3/52
P(H ∪ F) = 13/52 + 12/52 - 3/52 = 22/52 = 11/26
Multiplication Law
For any two events A and B:
P(A and B) = P(A ∩ B) = P(A) × P(B|A)
If A and B are independent (the occurrence of one does not affect the other), then:
P(A ∩ B) = P(A) × P(B)
Example: Multiplication Law
When drawing two cards from a standard deck without replacement, what is the probability of getting two aces?
Let A₁ = "first card is an ace" and A₂ = "second card is an ace"
P(A₁) = 4/52 (There are 4 aces in a deck of 52 cards)
P(A₂|A₁) = 3/51 (After drawing an ace, there are 3 aces left out of 51 remaining cards)
Using the multiplication rule:
P(A₁ ∩ A₂) = P(A₁) × P(A₂|A₁) = 4/52 × 3/51 = 12/2652 = 1/221
Complement Rule
For any event A:
P(A') = 1 - P(A)
where A' (or Aᶜ) is the complement of A (i.e., A does not occur)
Example: Complement Rule
What is the probability of rolling at least one 6 in four rolls of a fair die?
It's easier to calculate the complement: the probability of not rolling any 6 in four rolls.
P(no 6 in one roll) = 5/6
P(no 6 in four rolls) = (5/6)⁴ ≈ 0.482
Using the complement rule:
P(at least one 6) = 1 - P(no 6 in four rolls) = 1 - (5/6)⁴ ≈ 1 - 0.482 = 0.518
So the probability of rolling at least one 6 in four rolls is about 0.518 or 51.8%.
Bayes' Theorem
For events A and B with P(B) > 0:
P(A|B) = [P(B|A) × P(A)] / P(B)
Example: Bayes' Theorem
A manufacturing plant has three machines (A, B, and C) that produce 20%, 30%, and 50% of the items, respectively. The defect rates for these machines are 5%, 3%, and 2%. If an item is selected at random and found to be defective, what is the probability that it was produced by machine A?
Let's define the events:
MA, MB, MC = item was produced by machine A, B, or C
D = item is defective
We need to find P(MA|D).
Given information:
P(MA) = 0.20, P(MB) = 0.30, P(MC) = 0.50
P(D|MA) = 0.05, P(D|MB) = 0.03, P(D|MC) = 0.02
Using Bayes' theorem:
P(MA|D) = [P(D|MA) × P(MA)] / P(D)
We need to calculate P(D) using the law of total probability:
P(D) = P(D|MA) × P(MA) + P(D|MB) × P(MB) + P(D|MC) × P(MC)
P(D) = 0.05 × 0.20 + 0.03 × 0.30 + 0.02 × 0.50 = 0.01 + 0.009 + 0.01 = 0.029
Now we can calculate:
P(MA|D) = (0.05 × 0.20) / 0.029 = 0.01 / 0.029 ≈ 0.345
Therefore, there's approximately a 34.5% probability that a defective item was produced by machine A.
Probability Distributions
Probability distributions describe how the probabilities are distributed over the values of a random variable.
Discrete Probability Distributions
Binomial Distribution
Used for scenarios with a fixed number of independent trials, each with two possible outcomes ("success" or "failure").
The probability of x successes in n trials is:
P(X = x) = nCx × px × (1-p)n-x
where p is the probability of success on a single trial and nCx is the binomial coefficient.
Example: Binomial Distribution
A basketball player makes 80% of her free throws. What is the probability that she makes exactly 4 out of 5 free throws?
Here, n = 5, x = 4, p = 0.8
First, calculate 5C4 = 5! / [4! × (5-4)!] = 5! / (4! × 1!) = 5/1 = 5
Now apply the binomial formula:
P(X = 4) = 5C4 × 0.84 × (1-0.8)5-4 = 5 × 0.84 × 0.21
P(X = 4) = 5 × 0.4096 × 0.2 = 5 × 0.08192 = 0.4096
The probability is 0.4096 or approximately 41%.
Poisson Distribution
Used to model the number of events occurring in a fixed interval of time or space, given that these events occur with a known constant mean rate.
The probability of x occurrences is:
P(X = x) = (e-λ × λx) / x!
where λ is the average number of occurrences in the interval.
Example: Poisson Distribution
A call center receives an average of 5 calls per hour. What is the probability of receiving exactly 3 calls in an hour?
Here, λ = 5, x = 3
Using the Poisson formula:
P(X = 3) = (e-5 × 53) / 3!
P(X = 3) = (e-5 × 125) / 6 ≈ (0.00674 × 125) / 6 ≈ 0.8425 / 6 ≈ 0.1404
The probability is approximately 0.1404 or 14.04%.
Continuous Probability Distributions
Normal (Gaussian) Distribution
The most common continuous probability distribution, characterized by a bell-shaped curve.
The probability density function (PDF) is:
f(x) = (1 / (σ√(2π))) × e-((x-μ)2/(2σ2))
where μ is the mean and σ is the standard deviation.
Example: Normal Distribution
The heights of adult women in a country follow a normal distribution with a mean of 165 cm and a standard deviation of 6 cm. What proportion of women are taller than 175 cm?
First, standardize the value by converting to a z-score:
z = (x - μ) / σ = (175 - 165) / 6 = 10 / 6 ≈ 1.67
Using a standard normal table or calculator, we find that P(Z > 1.67) ≈ 0.0475
Therefore, approximately 4.75% of women are taller than 175 cm.
Uniform Distribution
A continuous distribution where all outcomes in a finite interval are equally likely.
For a uniform distribution on the interval [a, b], the probability density function is:
f(x) = 1 / (b - a) for a ≤ x ≤ b
Methods for Solving Probability Problems
1. Sample Space Analysis
Identify all possible outcomes (the sample space) and count favorable outcomes.
Example: Sample Space Analysis
What is the probability of rolling a sum of 8 with two fair dice?
Sample space: All possible outcomes when rolling two dice = 6 × 6 = 36 outcomes
Favorable outcomes: (2,6), (3,5), (4,4), (5,3), (6,2) = 5 outcomes
P(sum of 8) = 5/36 ≈ 0.139 or 13.9%
2. Tree Diagrams
Visual representation of all possible outcomes in a sequence of events. Useful for calculating conditional probabilities.
Example: Tree Diagram
A bag contains 3 red and 2 blue marbles. Two marbles are drawn without replacement. What is the probability of drawing a red marble followed by a blue marble?
First draw:
P(red) = 3/5, P(blue) = 2/5
Second draw (given first draw was red):
P(blue|red) = 2/4 = 1/2
Using the multiplication rule:
P(red then blue) = P(red) × P(blue|red) = 3/5 × 1/2 = 3/10 = 0.3 or 30%
3. Venn Diagrams
Visual representation of sets and their relationships. Useful for problems involving unions and intersections.
Example: Venn Diagram
In a group of 60 students, 30 study mathematics, 35 study physics, and 15 study both subjects. What is the probability that a randomly selected student studies either mathematics or physics?
Let M = "student studies mathematics" and P = "student studies physics"
Given:
n(M) = 30, n(P) = 35, n(M ∩ P) = 15, total n = 60
Using the formula for union:
n(M ∪ P) = n(M) + n(P) - n(M ∩ P) = 30 + 35 - 15 = 50
P(M ∪ P) = n(M ∪ P) / n = 50/60 = 5/6 ≈ 0.833 or 83.3%
4. Combinatorial Analysis
Using counting techniques such as permutations and combinations to solve probability problems.
Key Formulas:
Permutations (order matters):
P(n,r) = n! / (n-r)!
Combinations (order doesn't matter):
C(n,r) = n! / [r! × (n-r)!]
Example: Combinatorial Analysis
In a lottery, 6 numbers are drawn from 49 numbers. What is the probability of matching all 6 numbers?
Total number of possible ways to select 6 numbers from 49:
C(49,6) = 49! / [6! × (49-6)!] = 49! / [6! × 43!] = 13,983,816
Number of ways to match all 6 numbers: 1
P(matching all 6) = 1 / 13,983,816 ≈ 0.0000000715 or about 1 in 14 million
5. Expected Value
A weighted average of all possible outcomes, where each outcome is weighted by its probability.
For a discrete random variable X with possible values x1, x2, ..., xn and corresponding probabilities p1, p2, ..., pn:
E(X) = x1p1 + x2p2 + ... + xnpn
Example: Expected Value
In a game, you roll a fair die. If you roll a 1 or 2, you win $5. If you roll a 3 or 4, you lose $2. If you roll a 5 or 6, you win $1. What is the expected value of this game?
Let X be the amount won or lost.
X = $5 with probability 2/6 = 1/3
X = -$2 with probability 2/6 = 1/3
X = $1 with probability 2/6 = 1/3
E(X) = $5 × (1/3) + (-$2) × (1/3) + $1 × (1/3) = $5/3 - $2/3 + $1/3 = $4/3 ≈ $1.33
The expected value is $1.33, meaning on average, you would expect to win $1.33 per play of this game.
Practical Examples with Step-by-Step Solutions
Example 1: Deck of Cards
From a standard deck of 52 cards, what is the probability of drawing a red king or any queen?
Step 1: Define the events.
Let A = "drawing a red king" and B = "drawing any queen"
Step 2: Find the probability of each event.
P(A) = 2/52 = 1/26 (There are 2 red kings out of 52 cards)
P(B) = 4/52 = 1/13 (There are 4 queens out of 52 cards)
Step 3: Determine the intersection of the events.
P(A ∩ B) = 0 (A card cannot be both a king and a queen)
Step 4: Apply the addition rule.
P(A ∪ B) = P(A) + P(B) - P(A ∩ B) = 1/26 + 1/13 - 0 = 1/26 + 2/26 = 3/26 ≈ 0.115
Therefore, the probability of drawing a red king or any queen is 3/26, or approximately 11.5%.
Example 2: Conditional Probability
A company has two factories, A and B, producing light bulbs. Factory A produces 60% of the bulbs, and Factory B produces 40%. Factory A's defect rate is 3%, while Factory B's defect rate is 5%. If a bulb is selected at random and found to be defective, what is the probability it was produced by Factory B?
Step 1: Define the events.
Let FA = "bulb produced by Factory A"
Let FB = "bulb produced by Factory B"
Let D = "bulb is defective"
Step 2: Identify the given probabilities.
P(FA) = 0.6, P(FB) = 0.4
P(D|FA) = 0.03, P(D|FB) = 0.05
Step 3: We need to find P(FB|D) using Bayes' theorem.
P(FB|D) = [P(D|FB) × P(FB)] / P(D)
Step 4: Calculate P(D) using the law of total probability.
P(D) = P(D|FA) × P(FA) + P(D|FB) × P(FB)
P(D) = 0.03 × 0.6 + 0.05 × 0.4 = 0.018 + 0.02 = 0.038
Step 5: Apply Bayes' theorem.
P(FB|D) = (0.05 × 0.4) / 0.038 = 0.02 / 0.038 ≈ 0.526
Therefore, if a bulb is selected at random and found to be defective, there is approximately a 52.6% probability that it was produced by Factory B.
Example 3: Binomial Probability
A new drug has an 85% success rate for treating a disease. If 10 patients with the disease are treated with this drug, what is the probability that at least 8 of them will recover?
Step a: Identify the scenario as a binomial problem.
This is a binomial probability problem with n = 10 trials, p = 0.85 (probability of success), and we need to find P(X ≥ 8).
Step 2: Express the probability in terms of the binomial distribution.
P(X ≥ 8) = P(X = 8) + P(X = 9) + P(X = 10)
Step 3: Calculate each term using the binomial formula.
P(X = x) = nCx × px × (1-p)n-x
P(X = 8) = 10C8 × 0.858 × 0.152 = 45 × 0.858 × 0.152 ≈ 45 × 0.2725 × 0.0225 ≈ 0.2757
P(X = 9) = 10C9 × 0.859 × 0.151 = 10 × 0.859 × 0.15 ≈ 10 × 0.2316 × 0.15 ≈ 0.3474
P(X = 10) = 10C10 × 0.8510 × 0.150 = 1 × 0.8510 × 1 ≈ 0.1969
Step 4: Sum the individual probabilities.
P(X ≥ 8) = 0.2757 + 0.3474 + 0.1969 = 0.82
Therefore, the probability that at least 8 out of 10 patients will recover is approximately 0.82 or 82%.