Bivariate normal distribution and its distribution function as correlation coefficient $\rightarrow \pm 1$

I am not sure what happens to a bivariate normal distribution when $|\rho| \rightarrow 1$. Is the distribution well defined then? Moreover, when $$\Phi \left(\frac{x_1}{\sigma_1}, \frac{x_2}{\sigma_2}, \rho \right) = 1 \text{ or } 0?$$ Namely, is it true that if $\Bigl \lvert \frac{x_1}{\sigma_1} \Bigr \lvert \leq M$ and $\Bigl \lvert \frac{x_2}{\sigma_2} \Bigr \lvert \leq L$, where $M, L < \infty$, then $$\Phi \left(\frac{x_1}{\sigma_1}, \frac{x_2}{\sigma_2}, \rho \right) \neq 1 \text{ and } 0?$$ Can it be stated without any restrictions on $\rho$? If you know any reference, where it is explained well, I would be very grateful.

• I am not sure what exactly your notation means, but the limit of the joint cumulative probability distribution function is not restricted to only two values $0$ and $1$ but should be a nondecreasing function of both variables like $$F(x,y) = \Phi\left(\min(x, y)\right),$$ being an increasing function of $x$ for $x < y$ that becomes the constant $\Phi(y)$ in the region $x > y$, and similarly for fixed $x$ and varying $y$. Here, $\Phi(\cdot)$ is the univariate standard normal CDF and I have assumed that $X$ and $Y$ are marginally standard normal for simplicity. – Dilip Sarwate Jul 17 '13 at 11:03
• @DilipSarwate: I didn't get your argument. Let $\Sigma$ be a variance-covariance matrix. If $\Sigma$ is positive definite and $\| \Sigma \| < \infty$ (the norm is smaller than infinity, implying that all elements are smaller than infinity, thus the distribution function is well defined), then $\lim_{x_1 \rightarrow - \infty} \Phi_2(x_1,x_2;\Sigma) = 0$? – Kolibris Jul 17 '13 at 11:22
• You are correct. I stated the distribution function when $\rho = 1$ (as does the answer by @TooTone which also gives the result for $\rho = -1$) but the result that TooTone and I stated is not necessarily the limit of $\Phi(x,y;\rho)$ as $\rho$ approaches $1$: $$\lim_{\rho \to 1} \Phi_2(x,y; \rho) ~\text{does not necessarily equal}~ \Phi_1(\min(x,y)).$$ – Dilip Sarwate Jul 17 '13 at 13:43
• @DilipSarwate Good point. I'm afraid that subtleties like that are beyond my pay-grade! – TooTone Jul 17 '13 at 14:04
• @Dilip Nevertheless--understanding your "$\Phi_2$" and "$\Phi_1$" to be the CDFs for arbitrary $\rho$ and $\rho=1$, respectively--the (pointwise) limiting value of the former does equal the latter. – whuber Jul 17 '13 at 14:31

Yes, it's well-defined. For convenience and ease of exposition I'm changing your notation to use two standard normal distributed random variables $X$ and $Y$ in place of your $X1$ and $X2$. I.e., $X = (X1 - \mu_1)/\sigma_1$ and $Y = (X2 - \mu_2)/\sigma_2$. To standardize you subtract the mean and then divide by the standard deviation (in your post you only did the latter).

When $\rho=1$, the variables are perfectly correlated (which in the case of a normal distribution means perfectly dependent), so $X=Y$. And when $\rho=-1$, $X=-Y$.

The cumulative distribution function $\Phi(x,y,\rho)$ is defined to be the probability $\rm{Pr}(X\le x \cap Y\le y)$ given the correlation $\rm{Corr}(X,Y)=\rho$. I.e., $X$ has to be $\le x\,$ AND $\;Y$ has to be $\le y$.

So in the case $\rho=1$, $$\begin{eqnarray*} \Phi(x,y) &=& \rm{Pr}(X\le x \cap Y\le y) \\ &=& \rm{Pr}(X\le x \cap X\le y) \\ &=& \rm{Pr}(X\le \rm{min}(x,y)) \\ &=& \Phi_X(\rm{min}(x,y)) \\ &=& \Phi_Y(\rm{min}(x,y)). \\ \end{eqnarray*}$$

Here, $0 < \Phi(x,y) < 1$, so long as $|x|, |y| < \infty$.

The case $\rho=-1$ is much more interesting (and I'm not 100% sure I've got this right so would welcome corrections):

$$\begin{eqnarray*} \Phi(x,y) &=& \rm{Pr}(X\le x \cap Y\le y) \\ &=& \rm{Pr}(X\le x \cap -X\le y) \\ &=& \rm{Pr}(X\le x \cap X \ge -y) \\ &=& \rm{Pr}(-y \le X \le x) \\ &=& \Phi_X(x) - \Phi_X(-y) \;\;\;\mbox{(*)}\\ &=& \Phi_X(x) - (1 - \Phi_X(y)) \\ &=& \Phi_X(x) + \Phi_X(y) -1. \end{eqnarray*}$$

Note that the step marked * assumes $-y < x$, or equivalently, $y > -x$. If this doesn't hold, then $\Phi = 0$.

Here, $0 \le \Phi(x,y) < 1$, so long as $|x|, |y| < \infty$. Compared to the case of $\rho=1$, it is now possible to get $\Phi(x,y) = 0$ with finite values of $x$ and $y$. E.g. if $x=1$ and $y=-2$, it's impossible to get both $X\le x$ and $X\ge y$ ($X\le 1$ and $X\ge 2$).

To get some intuition for how the cumulative distribution functions look, I've plotted 3d plots and contour plots for the two cases below.    R code for these plots

grid = expand.grid(x=seq(-3,3,0.05), y=seq(-3,3,0.05))
grid$phi1 = with(grid, pnorm(pmin(x,y))) grid$phi2 = with(grid, ifelse(-y<x,pnorm(x) + pnorm(y) -1,0))

library(lattice)
wireframe(data=grid, phi1 ~ x*y, shade=TRUE, main="X=Y", scales=list(arrows=FALSE))
contourplot(data=grid, phi1 ~ x*y, main="X=Y")
wireframe(data=grid, phi2 ~ x*y, shade=TRUE, main="X=-Y", scales=list(arrows=FALSE))
contourplot(data=grid, phi2 ~ x*y, main="X=-Y", cuts=10)

There are plenty of web pages which cover the bivariate standard normal distribution. Which one you find best is going to be dependent on you. I had a quick search and rather liked the following: http://webee.technion.ac.il/people/adler/lec36.pdf, as it has some nice diagrams on p8 of what happens as $\rho \rightarrow \pm 1$. In the case of $\rho = \pm 1$, plotting $X$ against $Y$ will give you a straight line through the origin, either $y=\pm x$. If you plot this yourself, you should get a good intuition as to why $\rm{min}$ occurs in the formula for $\rho =1$.

• Thank you very much, one problem is solved. I would be very happy if someone could help me with the second part of the question as well. – Kolibris Jul 17 '13 at 12:31
• @Kolibris no worries it turned out to be an interesting exercise. Sorry can you clarify exactly what the second part of the question is? – TooTone Jul 17 '13 at 12:38
• +1, the graphics are a very nice feature! How did you draw them? – COOLSerdash Jul 17 '13 at 12:41
• @COOLSerdash glad you like them! The wireframe plots are nicest I think, as you can see the "ridge" on the $X=Y$ plot, and the triangle of $\Phi=0$ on the $X=-Y" plot. I've added R code (which I checked properly this time!) – TooTone Jul 17 '13 at 12:48 • @TooTone: I will reformulate it a bit. Is it true that if I assume that$\| \Sigma \| < \infty$,$|x| < \infty$,$|y| < \infty|$, then$0<\Phi(x,y;\Sigma)<1\$? – Kolibris Jul 17 '13 at 12:50