Sums of Independent Random Variables; Expectation of Discrete Random Variables

Definition

A discrete random variable is a random variable that takes a countable set of values.

The expectation or expected value of a discrete random variable $X$ is the weighted average of all possible values of $X$, where each value is weighted by its probability:

$$E(X) = \sum x \, P(X=x)$$

provided the sum exists.

If $X$ and $Y$ are independent random variables, then the sum $Z = X + Y$ is also a random variable whose possible values come from adding values of $X$ and $Y$. If $X$ and $Y$ are independent, then the probability of their joint occurrence satisfies:

$$P(X=x, Y=y) = P(X=x)\,P(Y=y)$$

Also, expectation is linear, which means:

$$E(X+Y) = E(X) + E(Y)$$

whether or not $X$ and $Y$ are independent.

---

Main Content

1. Expectation of a Discrete Random Variable

Meaning of expectation

• : The expectation of a discrete random variable represents the average value we would expect if the random experiment were repeated many times under the same conditions. It is not always one of the actual values the variable can take; instead, it is a long-run average outcome.

Formula and interpretation

• : If a discrete random variable $X$ takes values $x_1, x_2, x_3, \dots$ with probabilities $p_1, p_2, p_3, \dots$, then:

$$E(X) = \sum_{i} x_i p_i$$

This is called the weighted mean because values with higher probabilities contribute more to the average.

Example 1: Simple die roll
Let $X$ be the outcome of a fair die. Then

$$E(X) = \frac{1+2+3+4+5+6}{6} = \frac{21}{6} = 3.5$$

The value $3.5$ is not possible on a die, but it is the average outcome over many rolls.

Example 2: Unequal probabilities
Suppose $X$ takes values $0,1,2$ with probabilities $0.2, 0.5, 0.3$. Then

$$E(X) = 0(0.2) + 1(0.5) + 2(0.3) = 0 + 0.5 + 0.6 = 1.1$$

So the expected value is $1.1$.

---

2. Sum of Independent Random Variables

Formation of a sum

• : If two random variables $X$ and $Y$ are given, their sum is a new random variable:

$$Z = X + Y$$

The value of $Z$ depends on the pair of outcomes from $X$ and $Y$. To find the distribution of $Z$, we often combine probabilities of all pairs whose sum is the same.

Role of independence

• : Independence means knowing the value of one random variable does not affect the probabilities of the other. This makes it possible to multiply probabilities when finding joint outcomes:

$$P(X=x, Y=y) = P(X=x)P(Y=y)$$

This property is very useful in finding the probability distribution of the sum.

Example: Sum of two fair dice
Let $X$ and $Y$ be the outcomes of two independent fair dice. Then $Z=X+Y$ can take values from 2 to 12.

For example:

• $Z=2$ happens only if $(1,1)$
• $Z=3$ happens if $(1,2)$ or $(2,1)$
• $Z=7$ happens in 6 ways:
  $(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)$

Since each pair has probability $1/36$, the probability of $Z=7$ is:

$$P(Z=7)=\frac{6}{36}=\frac{1}{6}$$

Useful idea: convolution for discrete sums
When $X$ and $Y$ are independent discrete random variables, the probability mass function of $Z=X+Y$ is obtained by adding probabilities of all pairs that give the same sum. This process is called convolution.

---

3. Linearity of Expectation and Its Use in Sums

Main theorem

• : One of the most important results in probability is:

$$E(X+Y) = E(X) + E(Y)$$

More generally,

$$E(X_1 + X_2 + \cdots + X_n) = E(X_1)+E(X_2)+\cdots+E(X_n)$$

This is true regardless of whether the random variables are independent.

Why it matters

• : This property makes expectation much easier to compute than a full distribution. Instead of finding every possible value of a sum and its probability, we can simply add the expected values of the components.

Example 1: Two dice
Let $X$ and $Y$ be the outcomes of two fair dice. Since each die has expectation $3.5$,

$$E(X+Y)=E(X)+E(Y)=3.5+3.5=7$$

So the average sum of two fair dice is 7.

Example 2: Multiple random variables
If a game gives prizes $X_1, X_2, X_3$ from three independent stages, then the total expected prize is:

$$E(X_1+X_2+X_3)=E(X_1)+E(X_2)+E(X_3)$$

This is true even if the stages are not independent, making expectation extremely powerful in applications.

Important distinction

Finding expectation of a sum

• is easy because of linearity.

Finding the full distribution of a sum

• is usually harder and often requires convolution.

---

Working / Process

1. Identify the random variable and its values

Determine whether the problem asks for expectation, the sum of variables, or the distribution of the sum. List all possible values each variable can take.

2. Use the correct formula

For expectation of a discrete variable:

$$E(X)=\sum xP(X=x)$$

For the sum of independent variables, use:

• direct addition of expectations for mean value,
• multiplication of probabilities for joint outcomes,
• convolution if the full distribution of the sum is required.

3. Compute and interpret the result

After calculating, explain what the answer means in context. For example, an expected value represents a long-run average, while a sum distribution tells how likely each total outcome is.

Process illustration for two independent dice

Die 1 outcomes: 1 2 3 4 5 6
Die 2 outcomes: 1 2 3 4 5 6

Add corresponding pairs:
(1,1)->2, (1,2)->3, ..., (6,6)->12

This helps show how sums are formed from all outcome pairs.

---

Advantages / Applications

Simplifies large problems

• : Expectation can be found without calculating every possible combined outcome. This is especially useful when dealing with many random variables.

Useful in real-life modeling

• : These concepts are used in insurance, finance, quality control, queueing systems, reliability engineering, and decision-making under uncertainty.

Foundation for advanced topics

• : Understanding sums of independent random variables and expected value is essential for studying variance, probability distributions of totals, random walks, and the Central Limit Theorem.

---

Summary

• The expectation of a discrete random variable is its weighted average value.
• The sum of independent random variables combines outcomes from separate random experiments.
• Expected values add neatly, which makes total averages easy to compute.

Important terms to remember

• Discrete random variable
• Expected value
• Independence
• Sum of random variables
• Linearity of expectation