Independent Random Variables

Definition

Two random variables $X$ and $Y$ are independent if, for every pair of sets of possible values $A$ and $B$,

$$P(X \in A,\; Y \in B) = P(X \in A)\,P(Y \in B)$$

In words, the probability that $X$ falls in a set $A$ and $Y$ falls in a set $B$ at the same time is equal to the product of their separate probabilities.

For discrete random variables, independence can also be stated as:

$$P(X=x,\;Y=y)=P(X=x)\,P(Y=y)$$

for all possible values $x$ and $y$.

For continuous random variables with joint density $f_{X,Y}(x,y)$, independence means:

$$f_{X,Y}(x,y)=f_X(x)\,f_Y(y)$$

for all $x$ and $y$, where $f_X$ and $f_Y$ are the marginal densities.

A very important point is that independence is different from zero correlation. Two variables can have correlation 0 and still not be independent.

---

Main Content

1. Independence of Events vs Independence of Random Variables

Independence of events

• is the basic idea: two events $A$ and $B$ are independent if $$P(A\cap B)=P(A)P(B)$$ Random variables extend this idea to the values they can take. Instead of asking whether two single events are independent, we ask whether all events generated by the random variables are independent.

Random variable independence is stronger and more complete

• . If $X$ and $Y$ are independent random variables, then any event involving $X$ is independent of any event involving $Y$. For example, the event $\{X \le 3\}$ is independent of the event $\{Y > 5\}$. This makes independence very powerful in probability calculations.

Example

Suppose:

• $X$ = outcome of a fair coin toss, where $X=1$ for Heads and $X=0$ for Tails
• $Y$ = outcome of a fair die roll

Then: $$P(X=1)=\frac12,\qquad P(Y=6)=\frac16$$ If they are independent: $$P(X=1, Y=6)=\frac12 \cdot \frac16=\frac1{12}$$

This means knowing the coin result gives no information about the die result.

---

2. Joint, Marginal, and Conditional Probability

• The joint probability distribution gives probabilities for pairs of values $(X,Y)$. Independence is checked by comparing the joint distribution with the product of the marginals.
• The marginal distribution of each random variable is its distribution considered alone. If the joint distribution factors into marginals, the variables are independent.

Conditional Probability Connection

Independence can also be expressed using conditional probability:

$$P(X \in A \mid Y \in B)=P(X \in A)$$

whenever $P(Y \in B)>0$.
This says that once we know $Y$, the probability distribution of $X$ does not change.

Example Table for Discrete Variables

Consider two binary random variables $X$ and $Y$:

$X$	$Y$	$P(X,Y)$
0	0	0.25
0	1	0.25
1	0	0.25
1	1	0.25

Marginals: $$P(X=0)=0.5,\quad P(X=1)=0.5$$ $$P(Y=0)=0.5,\quad P(Y=1)=0.5$$

Check: $$P(X=1,Y=1)=0.25 = 0.5 \times 0.5$$ This holds for all pairs, so $X$ and $Y$ are independent.

Non-Independent Example

If the table were:

$X$	$Y$	$P(X,Y)$
0	0	0.5
0	1	0
1	0	0
1	1	0.5

Then $Y=X$, so knowing one gives the other exactly. These variables are not independent.

---

3. Key Properties and Consequences

Product rule for probabilities

• : For independent random variables, probabilities of combined outcomes multiply. This is often the main practical benefit of independence.

Functions of independent variables remain independent if applied separately

• : If $X$ and $Y$ are independent, then $g(X)$ and $h(Y)$ are also independent for suitable functions $g$ and $h$. This is useful when transforming random variables in applications.

Mean and variance results

• : Independence simplifies expected values and variances. In particular, $$E[XY]=E[X]E[Y]$$ when $X$ and $Y$ are independent and the expectations exist.

Also, $$\operatorname{Var}(X+Y)=\operatorname{Var}(X)+\operatorname{Var}(Y)$$ if $X$ and $Y$ are independent.

This formula is one of the most important applications of independence.

Why independence matters in variance

For any two random variables, $$\operatorname{Var}(X+Y)=\operatorname{Var}(X)+\operatorname{Var}(Y)+2\operatorname{Cov}(X,Y)$$ If they are independent, then $\operatorname{Cov}(X,Y)=0$, so the extra term disappears.

ASCII sketch for intuition

Relationship between independent variables:

X outcomes:   x1   x2   x3
              |    |    |
              v    v    v
Y outcomes:   y1   y2   y3

No outcome of X changes the probabilities for Y.

Relationship between dependent variables:

X outcome ---> affects ---> Y probabilities

---

Working / Process

1. Identify the random variables

• Determine what each random variable represents.
• Write down all possible values it may take.
• Example: $X$ could be the result of a coin toss, and $Y$ could be the result of a die roll.

2. Find the joint distribution and marginals

• Calculate or list $P(X=x,Y=y)$ for all possible pairs.
• Then find $P(X=x)$ and $P(Y=y)$ by summing or integrating over the other variable.
• This step is essential because independence is checked through the relationship between joint and marginal distributions.

3. Verify the independence condition

• Check whether $$P(X=x,Y=y)=P(X=x)P(Y=y)$$ for all values $x,y$ in the discrete case.

• For continuous variables, check whether $$f_{X,Y}(x,y)=f_X(x)f_Y(y)$$

• If the equality holds everywhere needed, the variables are independent; otherwise, they are not.

---

Advantages / Applications

Simplifies probability calculations

• : Independent random variables allow probabilities of combined events to be found by multiplication instead of more complicated joint reasoning.

Useful in modeling real-world systems

• : Many models in science, engineering, finance, and computer science assume independence as a starting point because it makes analysis manageable.

Supports statistical inference and theory

• : Independence is central in sampling, estimation, regression assumptions, hypothesis testing, and laws such as the law of large numbers and central limit theorem.

---

Summary

• Independent random variables do not influence each other’s probabilities.
• Independence means the joint probability equals the product of the marginal probabilities.
• This idea makes computations easier and is widely used in probability and statistics.
• Important terms to remember: random variable, joint distribution, marginal distribution, conditional probability, independence, covariance