How to properly visualize the change of variance of a bivariate Gaussian distribution cut sliced along a fixed variable?

Question

I'm trying to understand the basics of Gaussian Distribution. I struggle to visualice how the variance of the conditional probability of say P (Y|X) changes when X is fixed (given X and Y have a joint gaussian distribution). So, I have two pictures , in picture A from many sources shows that the variances dont change but the mean does. But it seems reasonable to me also to consider the conditional as a cut of de bivariate along one axis but now the variances does change, why is it so? I'm I wrongly thinking Picture B is the conditional?

Picture A

[Picture updated thanks to contribution]

Picture B

Cuts along one variable. It seems that the variance (spread of black lines) increases or decreases according to a fixed X.

Answer 1

In the spirit of my original post at What is the intuition behind conditional Gaussian distributions?, we should look at the situation and consider it geometrically.

Begin with the bottom illustration, which slices the bivariate density in the vertical direction (that is, through fixed values of $x$):

Here is a plot of the slices through the positive $x$ values (those closest to you in the previous image) on common axes:

In this example, because the correlation is negative, as $x$ increases (1) the center of the slice shifts leftwards and (2) the height decreases. The height is the original bivariate density. I use the original color scheme to emphasize this.

You want to visualize the conditional densities: these represent the distribution of the response, $y,$ for any given value of $x.$ They are obtained by scaling the slices in the vertical (density) direction to give them a unit area, which is a basic requirement of any probability density. Here is an example of scaling the slice through $x=1.5$ appearing at the very front edge of the original graphic:

The arrows show how each original point is moved to a new point. The colors now represent the (new) conditional density while the heights represent both the original bivariate density and the conditional density (ignoring the fact that the two densities use different units of measurement!).

The conditional variance describes the spread of the conditional density. Evaluate the spread by examining the rescaled densities. Here is the plot of the original slices, all rescaled into conditional densities:

(The color continues to depict the original bivariate density.)

It is now evident that (in this example) all the conditional distributions are translates of a common distribution. Clearly they all have the same spread, whence their variances are equal.

Answer 2

Suppose that $X$ and $Y$ are jointly normal with correlation coefficient $\rho \in (-1,1)$ and identical marginal $\mathcal N(0,1)$ distributions. Then, the joint density is $$f_{X,Y}(x,y) = \frac{1}{2\pi \sqrt{1-\rho^2}}\exp\left[-\left.\left.\frac{1}{2(1-\rho^2)}\right(x^2-2\rho xy + y^2\right)\right]\tag{1}$$ and for any fixed value $x_0$ for $X$, we have that $$f_{X,Y}(x_0,y) = \frac{1}{2\pi \sqrt{1-\rho^2}}\exp\left[-\left.\left.\frac{1}{2(1-\rho^2)}\right(x_0^2-2\rho x_0y + y^2\right)\right].\tag{2}$$ The OP has drawn pictures of Eq. $(2)$ for different values of $x_0$ and seems to think that these cross-sections of the joint pdf solid are the various conditional densities of $Y$ given that $X$ has those specific values. He seems to think that the variances of these conditional densities increase as $|x_0|$ increases because the cross-sections seem more spread out. But, $f_{Y\mid X = x_0}(y\mid X=x_0)$, the conditional density of $Y$ given that $X=x_0$, is not given by $(2)$. In fact, $f_{X,Y}(x_0,y)$ is a valid pdf only in rare circumstances. Note that, in general and without any assumptions about normality etc, the integral $\int_{-\infty}^\infty f_{X,Y}(x_0,y) \,\mathrm dy$ equals $f_X(x_0)$ and so while $f_{X,Y}(x_0,y)$ is indeed nonnegative (as all pdfs must be), the area under the curve does not equal $1$ except when $x_0$ is such that $f_X(x_0)$ serendipitously happens to have value $1$. Note that $\dfrac{f_{X,Y}(x_0,y)}{f_X(x_0)}$, which we recognize immediately as the formula for the conditional pdf of $Y$ given that $X$ has value $x_0$, is indeed as valid pdf. Thus, \begin{align} f_{Y\mid X = x_0}(y\mid X=x_0) &= \dfrac{f_{X,Y}(x_0,y)}{f_X(x_0)}\\ &= \dfrac{\frac{1}{2\pi \sqrt{1-\rho^2}}\exp\left[-\left.\left.\frac{1}{2(1-\rho^2)}\right(x_0^2-2\rho x_0y + y^2\right)\right]}{\frac{1}{ \sqrt{2\pi}}\exp\left[-\left.\left.\frac{1}{2}\right(x_0^2\right)\right]}\\ &= \frac{1}{\sqrt{1-\rho^2}\sqrt{2\pi}}\exp\left[-\left.\left.\frac{1}{2}\right(\frac{y-\rho x_0}{\sqrt{1-\rho^2}}\right)^2\right] \end{align} which shows that the conditional density of $Y$ given that $X=x_0$ is a normal density with fixed variance $1-\rho^2$ (not depending on the given value $x_0$ of $X$ at all) but with mean $\rho x_0$ which does vary with $x_0$. This is what several of the comments on the OP's question are trying to point out:

the cross-section of the joint pdf solid at $x_0$ is not the conditional density of $Y$ given $X=x_0$ (though it is related). The conditional density has fixed variance (not depending on $x_0$) but variable mean (depending on $x_0$).

It is possible to read too much into the aphorism that the bivariate normal distribution is like a piece of bologna in the sense no matter how you slice it, it is still bologna. Every cross-section (not necessarily parallel to the axes) of the bivariate normal density solid is proportional to a normal density but is not exactly a normal density unless the cross-section passes through the mean point of the bivariate distribution.

Answer 3

If I plot a 95% ellipse for a bivariate normal distribution, it might look like the plot below.

When I take a vertical slice through that distribution, I might get a slice like the lighter red line, a, in the center, or I might get a slice like the darker red line, b, on the right. It looks like a is longer than b (and it is). That is, the vertical spread from the bottom to the top of the ellipse at a given point on $X$ changes as you move from side to side. This is simply not the same as the conditional variance. As a first attempt to understand the bivariate normal and the idea of homoscedasticity, just looking at the vertical spread is reasonable, but it isn't quite right. It's simply not the case that the length of those lines are the variances of the conditional distributions of $Y$ given $X=x_i$.

With a bivariate normal, you will have more data, and more density in the center (either from side to side, or equally, from top to bottom) than at the extremes. Where there is more data, you will have more extreme values. For instance, only 5% will be $>|2|$ SD's from the mean; where you have 100 data, that would be 5, but where you have 1000 data, that will be 50. The result is that the density is higher with more data and it looks like it is spreading out further, but in both cases, the variance is the same. To illustrate, consider the very simple simulation below. Even though the standard deviation is the same, with more data, the value at the 2.5th percentile is further out:

set.seed(1)  # makes this reproducible
quantile(sort(rnorm( 200, mean=0, sd=1)), probs=.025)
#      2.5% 
# -1.641215 
set.seed(1)
quantile(sort(rnorm(2000, mean=0, sd=1)), probs=.025)
#      2.5% 
# -2.092716