Unit 5: Elements of Probability

1. Basic Concepts (Experiment, Sample Space, Event)

Probability is the measure of the likelihood that an event will occur. It is quantified as a number between 0 and 1.

Random Experiment

An experiment or process for which the outcome cannot be predicted with certainty, but all possible outcomes are known.

• Example: Tossing a coin, rolling a die, drawing a card from a deck.

Sample Space (S)

The set of all possible outcomes of a random experiment.

• Example (Rolling a die): S = {1, 2, 3, 4, 5, 6}
• Example (Tossing two coins): S = {HH, HT, TH, TT}

Event (E)

A subset of the sample space. It is a collection of one or more outcomes.

• Example (Rolling a die):
  • Event A (Getting an even number): A = {2, 4, 6}
  • Event B (Getting a number > 4): B = {5, 6}

2. Algebra of Events

Events can be combined using set operations.

• Event 'A or B' (A ∪ B): The event that *at least one* of A or B occurs.
  • A = {2, 4, 6}, B = {5, 6} => A ∪ B = {2, 4, 5, 6}
• Event 'A and B' (A ∩ B): The event that *both* A and B occur simultaneously.
  • A = {2, 4, 6}, B = {5, 6} => A ∩ B = {6}
• Event 'Not A' (A' or A^c): The complement of A. The event that A does *not* occur.
  • S = {1, 2, 3, 4, 5, 6}, A = {2, 4, 6} => A' = {1, 3, 5}
• Mutually Exclusive Events: Two events that cannot occur at the same time. They have no outcomes in common.
  • (A ∩ B) = ∅ (the empty set).
• Exhaustive Events: A set of events that covers the entire sample space.
  • A₁ ∪ A₂ ∪ ... = S

3. Definitions of Probability (Classical & Statistical)

1. Classical (or 'a priori') Definition

This definition assumes all outcomes in the sample space are equally likely.

If a random experiment has 'n' mutually exclusive, exhaustive, and equally likely outcomes, and 'm' of these outcomes are favorable to an event A:

• Example: P(Drawing a King from a 52-card deck) = 4 / 52 = 1 / 13.
• Limitation: Fails if outcomes are not equally likely (e.g., a biased coin) or if the sample space is infinite.

2. Statistical (or Empirical / Relative Frequency) Definition

This definition is based on relative frequency from performing an experiment many times.

If an experiment is repeated 'n' times (where 'n' is very large) and event A occurs 'f' times, the probability of A is the relative frequency.

• Example: If we toss a coin 10,000 times and get 5,030 heads, we estimate P(Heads) ≈ 0.503.

4. Addition Theorem of Probability

This theorem (or law) is used to find the probability of (A or B) occurring.

Statement (for *any* two events A and B):

The probability that at least one of the events A or B occurs is given by:

(We subtract P(A ∩ B) because it was counted twice - once in P(A) and once in P(B)).

Statement (for *Mutually Exclusive* events):

If A and B are mutually exclusive, they cannot happen together, so P(A ∩ B) = 0. The formula simplifies to:

5. Multiplication Theorem of Probability (Conditional Probability)

This theorem is used to find the probability of (A and B) occurring.

Conditional Probability

First, we must define Conditional Probability, P(A | B). This is read as "the probability of A, given that B has already occurred."

Statement (Multiplication Theorem):

By rearranging the conditional probability formula, we get the multiplication theorem:

(It can also be written as: P(A ∩ B) = P(A) * P(B | A))

In words: The probability of A and B both happening is the probability of B happening, *multiplied by* the probability of A happening *given that B has already happened*.

Statement (for *Independent* events):

Two events are independent if the occurrence of one does not affect the probability of the other.
In this case, P(A | B) = P(A).

The multiplication theorem simplifies to: