# Does $cov(x,y)=cov(x,x)=cov(y,y)$ imply $x=y$?

Given two random variables, you can calculate their covariance matrix. I noticed that if I plot data (in my case multi-variate normal) coming from a cov-matrix whose elements are all the same, e.g.

$$\begin{pmatrix} 100 & 100 \\ 100 & 100 \end{pmatrix}$$

You will get a straight line. e.g. in python:

data = np.random.multivariate_normal([0,0], [[100,100],[100,100]], 1000)
plt.scatter(data[:,0], data[:,1])


I wonder if this implies that $$x=y$$?

Also, can there be a case where $$cov(x,y) = cov(x,x) \ne cov(y,y)$$ and if so, is there any insight about what's going on there?

Not quite. As an example, take

$$X\sim\mathcal{N}(0, 100) \\ Y = X+3,$$

or, equivalently,

$$\left[\begin{array}{c} X \\ Y\end{array}\right]\sim\mathcal{N}\bigg(\left[\begin{array}{c} 0 \\ 3\end{array}\right], \left[\begin{array}{cc} 100 & 100 \\ 100 & 100\end{array}\right]\bigg).$$

Then we have $$Var(X) = Var(Y) = Cov(X, Y) = 100$$, but $$X$$ and $$Y$$ are not equal ($$Y$$ is always 3 larger than $$X$$).

Though this example shows that $$Var(X)=Var(Y)=Cov(X,Y)$$ does not imply $$X=Y$$, it does imply (with probability 1) that $$X$$ and $$Y$$ differ by some constant $$c$$. The covariance matrix can't be used to determine what that constant is -- you would need to know the means as well to learn the constant.

To your second question, it is quite common to have $$Var(X) = Cov(X, Y) \neq Var(Y)$$, and there's nothing particularly special about this situation. All that it represents is two random variables with different variances and a moderate amount of correlation. For instance, the following are two normal random variables with different variances and a correlation of 0.1.

$$\left[\begin{array}{c} X \\ Y\end{array}\right]\sim\mathcal{N}\bigg(\left[\begin{array}{c} 0 \\ 0\end{array}\right], \left[\begin{array}{cc} 1 & 1 \\ 1 & 100\end{array}\right]\bigg).$$

Hint: $$Var(X-Y)=Var(X)+Var(Y)-2cou(X,Y)$$

and $$Var(X-Y)=E\bigg((X-Y)-E(X-Y)\bigg)^2=0$$

so $$P\{(X-Y)-E(X-Y)=0\}=1$$ so $$X-Y$$ almost surly constant

I want to discuss why you observed all the data on the particular line that you did.

$$\newcommand{\1}{\mathbf 1}$$If you have a random variable $$X \sim \mathcal N(\mathbf 0, \Sigma)$$ where $$\Sigma$$ has all the same element, then $$\Sigma \propto \1\1^T$$ so you have a rank $$1$$ covariance matrix.

I'll consider the more general problem of what it means to have a multivariate Gaussian with a low-rank covariance matrix. Suppose $$X \sim\mathcal N_p(\mathbf 0, \Sigma)$$ and $$1 \leq \text{rank}(\Sigma) := r < p$$. We can factor $$\Sigma$$ as $$\Sigma = \tilde Q\tilde \Lambda \tilde Q^T$$ via the spectral theorem with $$\Lambda = \text{diag}(\lambda_1,\dots,\lambda_r, 0, \dots, 0)$$. This means that $$\Sigma$$ can actually be represented as $$\Sigma = Q\Lambda Q^T$$ where $$Q$$ is the first $$r$$ columns of $$\tilde Q$$ and $$\Lambda = \text{diag}(\lambda_1,\dots,\lambda_r)$$ contains the non-zero eigenvalues.

Let $$Z \sim \mathcal N_r(\mathbf 0, I)$$ and define $$Y = Q\Lambda^{1/2}Z.$$ $$Y$$ is a linear transformation of a Gaussian so it too is Gaussian, and $$\text{E}(Y) = \mathbf 0 \\ \text{Var}(Y) = Q\Lambda^{1/2}\Lambda^{1/2}Q^T = \Sigma$$ so $$Y \sim \mathcal N(\mathbf 0, \Sigma) \stackrel{\text d}= X$$.

This shows that we can think of $$X$$ as being generated by a low dimensional Gaussian that we map up into our high dimensional space, and this explains why there is still randomness but not over all of $$\mathbb R^p$$. In particular, $$X$$ is constrained to the $$r$$-dimensional column space of $$Q$$.

In your case we have $$\Sigma \propto \1\1^T$$ which means that $$r=1$$ and $$Q = p^{-1/2}\mathbf 1$$. This shows that $$X \in \text{span}(\1)$$ which is what you observed.

One final comment: these Gaussians with low-rank covariance matrices do not have pdfs in the usual sense because $$P(X \in \text{ColSpace}(Q)) = 1$$ yet the Lebesgue measure of $$\text{ColSpace}(Q)$$ is zero w.r.t. the Lebesgue measure on $$\mathbb R^p$$. This is one advantage to defining a multivariate Gaussian as a random variable where every linear combination is Gaussian since then there's no issue with zero determinant covariance matrices in the expression for the usual pdf.