# Does $\text{cov}(a_1' X, a_2' X) = 0$ imply $a_1 \cdot a_2 = 0$?

Let $$X$$ be a $$p$$-dimensional random vector with $$p$$ principal components $$y_1, y_2, \dots, y_p$$. By definition, a restriction put on the second principal component $$y_2 = a_2'X$$ is

$$\text{cov}(y_1, y_2) = \text{cov}(a_1' X, a_2' X) = 0$$

It is also known that $$a_i$$ are simply eigenvectors, so we also have

$$\forall i \ne j, a_i \cdot a_j = 0$$

I wonder if it's a coincidence in the context of PCA. In other words, does $$\text{cov}(a_1' X, a_2' X) = 0$$ imply $$a_1 \cdot a_2 = 0$$? I tried to prove it, but didn't get very far

\begin{aligned} 0 &= \text{cov}(a_1' X, a_2' X) \\ &= \text{cov}(\sum_{k=1}^p a_{1,k} X_k, \sum_{m=1}^p a_{2,m} X_m) \\ &= \sum_{k=1}^p a_{1,k} \sum_{m=1}^p a_{2,m} \text{cov}(X_k, X_m) \end{aligned}

Can I get $$a_1 \cdot a_2 = 0$$ from here?

• Geometrically, you are asking whether two vectors remain orthogonal after an arbitrary linear transformation. In light of this it's easy to find counterexamples in just $p=2$ dimensions. In fact, you are almost certain to find a counterexample just by randomly selecting any $2\times 2$ covariance matrix. – whuber May 12 '19 at 16:08

Consider a random walk model, $$X_t = X_{t-1} + Z_t$$ where $$Z_t \overset{iid}{\sim} \text{Normal}(0,1)$$. Also, assume that $$X_1 = Z_1$$ and has variance $$1$$. Take the random vector $$X = (X_1,X_2,X_3)'$$, it has a covariance matrix $$\Sigma = \left[\begin{array}{ccc} 1 & 1 & 1 \\ 1 & 2 & 2 \\ 1 & 2 & 3 \end{array}\right].$$ Now take $$a_1 = (-1,1,0)'$$ and $$a_2 = (0,-1,1)$$. You can see that $$\text{Cov}(a_1'X, a_2'X) = \text{cov}(Z_2, Z_3) = 0$$ but $$a_1'a_2 = -1$$.
Edit: just for completeness, consider the example that your mind might go to first. Consider a vector $$X$$ such that its covariance matrix is proportional to the identity matrix (i.e. $$\sigma^2 \mathbf{I}$$ with $$\sigma^2 >0$$). Then $$\text{Cov}(a_1'X, a_2'X) = \sigma^2 a_1' a_2$$ is zero if and only if $$a_1' a_2 = 0$$.