Conditional Densities

Definition

For two continuous random variables $X$ and $Y$, the conditional density of $Y$ given $X=x$ is defined as

$$f_{Y|X}(y|x) = \frac{f_{X,Y}(x,y)}{f_X(x)}, \quad \text{provided } f_X(x) > 0$$

where:

• $f_{X,Y}(x,y)$ is the joint density of $X$ and $Y$
• $f_X(x)$ is the marginal density of $X$

Similarly, the conditional density of $X$ given $Y=y$ is

$$f_{X|Y}(x|y) = \frac{f_{X,Y}(x,y)}{f_Y(y)}, \quad \text{provided } f_Y(y) > 0$$

A conditional density must satisfy the usual properties of a density in the variable being conditioned on:

• It is nonnegative: $f_{Y|X}(y|x) \ge 0$
• Its total area over all possible values of $y$ is 1: $$\int_{-\infty}^{\infty} f_{Y|X}(y|x)\,dy = 1$$

Main Content

1. Joint, Marginal, and Conditional Densities

• The joint density $f_{X,Y}(x,y)$ describes the combined behavior of two continuous random variables together. It gives the likelihood of $X$ and $Y$ occurring near a particular pair $(x,y)$.

• The marginal density is obtained by integrating out the other variable: $$f_X(x)=\int_{-\infty}^{\infty} f_{X,Y}(x,y)\,dy, \qquad f_Y(y)=\int_{-\infty}^{\infty} f_{X,Y}(x,y)\,dx$$ Once the marginal density is known, the conditional density can be computed by dividing the joint density by the marginal.

• Example: If $$f_{X,Y}(x,y)=2 \quad \text{for } 0<y<x<1$$ and 0 otherwise, then $$f_X(x)=\int_0^x 2\,dy = 2x,\quad 0<x<1$$ so $$f_{Y|X}(y|x)=\frac{2}{2x}=\frac{1}{x}, \quad 0<y<x<1$$ This means that, given $X=x$, the variable $Y$ is uniformly distributed on $(0,x)$.

2. Interpretation of Conditional Density

• A conditional density describes the distribution of one variable when the other variable is fixed. It answers questions such as: “If $X=x$, what values of $Y$ are most likely?”

• It is a continuous version of conditional probability, but because continuous probabilities at exact points are zero, the density is interpreted through neighborhoods and intervals rather than point probabilities. For a small interval $[y, y+\Delta y]$, $$P(y \le Y \le y+\Delta y \mid X=x) \approx f_{Y|X}(y|x)\Delta y$$

• Intuition diagram for a conditional slice:

  Joint density surface
        ^
        |          .
        |       .     .
        |    .           .
        | .                .
        +------------------------> x
         |
         |  Fix x = x0
         |
         |---- vertical slice gives f(Y|X=x0)

This shows that conditioning on $X=x_0$ means taking a “slice” of the joint distribution at that fixed value.

3. Properties and Use in Probability Calculations

• Conditional densities help compute conditional probabilities and expectations. For a set $A$, $$P(Y \in A \mid X=x)=\int_A f_{Y|X}(y|x)\,dy$$ and the conditional expectation is $$E[Y\mid X=x]=\int_{-\infty}^{\infty} y\,f_{Y|X}(y|x)\,dy$$

• They are essential for the law of total probability and law of total expectation in continuous settings: $$f_Y(y)=\int_{-\infty}^{\infty} f_{Y|X}(y|x)f_X(x)\,dx$$ $$E[Y]=E\big(E[Y\mid X]\big)$$

• Example: If $Y|X=x \sim \text{Uniform}(0,x)$ for $x>0$, then $$f_{Y|X}(y|x)=\frac{1}{x}, \quad 0<y<x$$ and $$E[Y|X=x]=\int_0^x y\cdot \frac{1}{x}\,dy=\frac{x}{2}$$ So, on average, $Y$ is half of $X$ when $X$ is fixed.

Working / Process

1. Start with the joint density

• Identify the joint density function $f_{X,Y}(x,y)$ and its support region.
• Make sure it is properly normalized so that the total integral over the support is 1.
• This step is crucial because the conditional density is built directly from the joint distribution.

2. Find the relevant marginal density

• Integrate the joint density over the other variable.
• For $f_{Y|X}(y|x)$, compute $f_X(x)$; for $f_{X|Y}(x|y)$, compute $f_Y(y)$.
• Verify that the marginal is positive where the conditional density is to be defined.

3. Divide and simplify

• Use $$f_{Y|X}(y|x)=\frac{f_{X,Y}(x,y)}{f_X(x)}$$ and simplify the expression.

• Check that the conditional density is nonnegative and integrates to 1 with respect to the conditioned variable.

4. Use the conditional density for further calculations

• Compute conditional probabilities by integrating over the relevant interval.
• Compute conditional expectations, variances, and predictive distributions.
• Apply the result to interpret dependence between variables.

Advantages / Applications

• Conditional densities provide a precise way to model dependence between continuous random variables, making them essential in multivariate statistics and probability theory.
• They are widely used in Bayesian inference, where posterior distributions are conditional densities of parameters given observed data.
• They are important in regression, machine learning, signal processing, and stochastic modeling, where future or unknown values are predicted using known information.

Summary

• Conditional density describes the distribution of one continuous variable given another variable.
• It is found by dividing the joint density by the relevant marginal density.
• It is used to compute conditional probabilities, expectations, and model dependence between variables.
• Important terms to remember: joint density, marginal density, conditional density, support, conditional expectation