Basic Probability: Probability Spaces

Definition

A probability space is an ordered triple
$$(\Omega, \mathcal{F}, P)$$ where:

$\Omega$

• is the sample space, the set of all possible outcomes of a random experiment.

$\mathcal{F}$

• is a sigma-algebra (or $\sigma$-algebra) of subsets of $\Omega$, called the collection of events.

$P$

• is a probability measure, a function that assigns a number between 0 and 1 to each event in $\mathcal{F}$, satisfying the axioms of probability.

Important components in detail

1. Sample Space $\Omega$

The sample space contains every possible outcome of the experiment.

Examples:

• Tossing a coin:
  $$\Omega = \{H, T\}$$

• Rolling a die:
  $$\Omega = \{1,2,3,4,5,6\}$$

• Tossing two coins:
  $$\Omega = \{HH, HT, TH, TT\}$$

2. Event Collection $\mathcal{F}$

An event is any set of outcomes we want to discuss. In many elementary cases, $\mathcal{F}$ may be the set of all subsets of $\Omega$, called the power set. In more advanced settings, not every subset is included, so we use a sigma-algebra.

A sigma-algebra must satisfy:

• The empty set $\emptyset$ is included.
• If an event is included, then its complement is also included.
• If countably many events are included, then their union is also included.

3. Probability Measure $P$

The probability measure assigns a likelihood to each event.

It must satisfy:

Non-negativity

• : $P(A) \ge 0$ for every event $A$.

Normalization

• : $P(\Omega)=1$.

Countable additivity

• : If $A_1, A_2, A_3, \dots$ are pairwise disjoint events, then $$P\left(\bigcup_{i=1}^{\infty} A_i\right)=\sum_{i=1}^{\infty} P(A_i)$$

Example: If a fair die is rolled, then each outcome has probability $1/6$, and the probability of getting an even number is: $$P(\{2,4,6\})= \frac{3}{6}=\frac{1}{2}$$

---

Main Content

1. Sample Space

• The sample space is the starting point of every probability model because it lists every possible result of the experiment.
• It may be finite, countably infinite, or uncountably infinite, depending on the experiment.

Types of sample spaces

Finite sample space:
A finite number of outcomes.

• Example: Tossing a coin once $$\Omega = \{H,T\}$$

Countably infinite sample space:
Outcomes can be listed one by one.

• Example: Number of tosses until the first head appears: $$\Omega = \{1,2,3,\dots\}$$

Uncountably infinite sample space:
Outcomes cannot be listed in a sequence.

• Example: Choosing a random number from the interval $[0,1]$.

Why the sample space matters

The sample space determines the universe of discussion. Every event must be a subset of it. If the sample space is chosen incorrectly, the whole probability model becomes flawed.

Example with two dice

If two dice are rolled, the sample space consists of ordered pairs: $$\Omega = \{(1,1),(1,2),\dots,(6,6)\}$$ There are $6 \times 6 = 36$ possible outcomes.

ASCII illustration of a simple sample space

For a coin toss:

        Coin Toss
           |
     ----------------
     |              |
     H              T

This diagram shows that there are only two possible outcomes.

---

2. Events and Sigma-Algebra

• An event is a set of outcomes of interest. For example, in a die roll, “getting an even number” is the event $\{2,4,6\}$.
• A sigma-algebra $\mathcal{F}$ is the collection of events for which probabilities are defined consistently.

Why not use all subsets always?

For finite sample spaces, all subsets can usually be used. But for infinite or continuous spaces, certain subsets are too complicated or lead to contradictions if probability is assigned to every possible subset. A sigma-algebra avoids this problem by selecting a well-behaved family of sets.

Properties of a sigma-algebra

A collection $\mathcal{F}$ of subsets of $\Omega$ is a sigma-algebra if:

1. $\emptyset \in \mathcal{F}$
2. If $A \in \mathcal{F}$, then $A^c \in \mathcal{F}$
3. If $A_1, A_2, A_3, \dots \in \mathcal{F}$, then $$\bigcup_{n=1}^{\infty} A_n \in \mathcal{F}$$

From these rules, it also follows that:

• $\Omega \in \mathcal{F}$
• intersections of countably many events are also in $\mathcal{F}$

Event examples

For a die:

• Event $A$: “even number” $$A = \{2,4,6\}$$

• Event $B$: “number greater than 4” $$B = \{5,6\}$$

• Event $A \cap B$: “even and greater than 4” $$A \cap B = \{6\}$$

Event relationships

Union

• : $A \cup B$ means either event occurs.

Intersection

• : $A \cap B$ means both occur.

Complement

• : $A^c$ means event $A$ does not occur.

Disjoint events

• : $A \cap B = \emptyset$, meaning they cannot happen together.

Example of an event structure

For rolling a die:

Ω = {1, 2, 3, 4, 5, 6}

A = {2, 4, 6}   (even numbers)
B = {4, 5, 6}   (greater than 3)

A ∩ B = {4, 6}
A ∪ B = {2, 4, 5, 6}
A^c = {1, 3, 5}

This structure is essential for combining and analyzing outcomes logically.

---

3. Probability Measure and Axioms

• The probability measure $P$ assigns a number to every event in the sigma-algebra.
• This number represents the chance of the event occurring, and it always lies between 0 and 1.

The three axioms of probability

Axiom 1: Non-negativity

For every event $A$, $$P(A) \ge 0$$ Probability cannot be negative.

Axiom 2: Normalization

$$P(\Omega)=1$$ The total probability of the entire sample space is 1 because one of the possible outcomes must occur.

Axiom 3: Countable additivity

If $A_1, A_2, A_3, \dots$ are pairwise disjoint events, then: $$P\left(\bigcup_{i=1}^{\infty} A_i\right)=\sum_{i=1}^{\infty}P(A_i)$$

This means probabilities of mutually exclusive events add up.

Immediate consequences

From the axioms, we can derive:

• $P(\emptyset)=0$
• $P(A^c)=1-P(A)$
• If $A \subseteq B$, then $P(A)\le P(B)$
• For any two events $A$ and $B$, $$P(A \cup B)=P(A)+P(B)-P(A \cap B)$$

Example: fair die

Let $A=\{2,4,6\}$, the event of getting an even number.

If the die is fair, then each outcome has probability $1/6$, so: $$P(A)=\frac{3}{6}=\frac{1}{2}$$

The complement is: $$A^c=\{1,3,5\}$$ and: $$P(A^c)=1-\frac{1}{2}=\frac{1}{2}$$

Example: union of events

Let:

• $A=\{2,4,6\}$
• $B=\{4,5,6\}$

Then: $$P(A)=\frac{3}{6}, \quad P(B)=\frac{3}{6}, \quad P(A\cap B)=\frac{2}{6}$$

So, $$P(A\cup B)=\frac{3}{6}+\frac{3}{6}-\frac{2}{6}=\frac{4}{6}=\frac{2}{3}$$

---

Working / Process

1. Define the random experiment

• Decide what process is being studied, such as tossing a coin, rolling a die, selecting a card, or measuring a random variable.
• The experiment must be clearly stated so that outcomes can be identified.

2. Construct the sample space and event collection

• List all possible outcomes in the sample space $\Omega$.
• Identify the relevant events and organize them into a sigma-algebra $\mathcal{F}$.
• For simple finite experiments, this may be all subsets of $\Omega$.

3. Assign probabilities using the probability measure

• Define $P$ so that each event gets a number between 0 and 1.
• Check the axioms:
  • probabilities are non-negative,
  • the whole sample space has probability 1,
  • disjoint events add correctly.
• Use the probability space to compute event probabilities, complements, unions, intersections, and more complex expressions.

Example workflow: rolling a die

1. Experiment: roll one fair die.

2. Sample space: $$\Omega=\{1,2,3,4,5,6\}$$

3. Event: getting a number greater than 4 $$A=\{5,6\}$$

4. Probability: $$P(A)=\frac{2}{6}=\frac{1}{3}$$

Visual workflow

Experiment → Sample Space → Events → Probability Measure → Probability Results

This process is the practical way to build and use a probability space.

---

Advantages / Applications

• Probability spaces provide a rigorous foundation for all of probability theory, making definitions and results mathematically precise.
• They help model uncertainty in real-world situations, such as dice, coins, lotteries, weather, traffic, and financial markets.
• They are essential in advanced topics like random variables, distributions, conditional probability, expected value, stochastic processes, and statistical inference.

More detailed applications

1. Games of chance Probability spaces are used to analyze card games, dice games, and lotteries. They help determine fair outcomes and winning chances.

2. Scientific experiments In physics and chemistry, random experiments and measurement uncertainty are modeled using probability spaces.

3. Engineering and computer science Reliability of systems, error rates, communication channels, randomized algorithms, and queueing systems all rely on probability spaces.

4. Data science and machine learning Probability spaces support the mathematical basis of classification, prediction, uncertainty quantification, and model evaluation.

5. Decision-making Businesses and economists use probability spaces to evaluate risk and make informed decisions under uncertainty.

---

Summary

• A probability space is the mathematical framework for describing random experiments.
• It consists of a sample space, a collection of events, and a probability measure.
• The probability axioms ensure that probabilities are consistent and logically valid.
• Important terms to remember: sample space, event, sigma-algebra, probability measure, and axioms of probability.