Show that $\text{Cov}(a^TX, e^T_i(X − \mu)) = \lambda_ia^Te_i$

Question

Suppose that the $p$-dimensional random vector $X = [X1,\dotsc,Xp]^T$ has mean $\mathbb{E}(X) = \mu$ and positive definite covariance matrix $\text{Cov}(X) = \Sigma = {\sigma_{ij}}$. Also suppose that $\Sigma$ has distinct eigenvalues $\lambda_1,\dotsc, \lambda_p$ where $\lambda_1 >\dotsb> \lambda_p > 0.$ Let $e_1,\dotsc,e_p$ denote the eigenvectors of unitary length corresponding to the eigenvalues $\lambda_1,..., \lambda_p,$ respectively. Also let the $j$’th element of $e_i$ be denoted $e_{ij}$.

Let $X = [X_1,\dotsc,X_p]^T$ be a random vector and let $c = [c_1,\dotsc,c_p]^T$ be a vector of constants.

1. The linear combination $c^T X = c_1 X_1 +\dotsb+ c_p X_p$ has $\mathbb{E}(c^T X) = c^T \mathbb{E}(X)$ and $\text{Var}(c^T X) = c^T \text{Cov}(X) c.$
2. $\text{Cov}(a^T X, c^T X) = a^T \text{Cov}(X) c.$

Using the above show that, for a constant vector a, $\text{Cov}(a^TX, e^T_i(X − µ)= \lambda_ia^Te_i.$

I'm just not sure how to prove this.
I know $\text{Var}(a^TX) = a^T\Sigma a$ and $\text{Var}(e^T_i(X − \mu)) = \lambda_i.$

Any suggestions?

Answer 1

Because of $\text{Cov}$ linearity you can write:

$$\text{Cov}(a^{T}X,e_{i}^{T}(X-\mu))=\text{Cov}(a^{T},e_{i}^{T}X)-\text{Cov}(a^{T}X,e_{i}^{T}\mu).$$

I am barely using your second formula and the definition of $\text{Cov}$:

$$\text{Cov}(a^{T}X,e_{i}^{T}(X-\mu))=a^{T}\text{Cov}(X)e_{i}-\mathbb{E}(a^{T}Xe_{i}^{T}\mu)-\mathbb{E}(a^{T}X)\mathbb{E}(e_{i}^{T}\mu).$$

I can move out of the expectation all fixed quantities and $\text{Cov}(X)=\Sigma$:

$$\text{Cov}(a^{T}X,e_{i}^{T}(X-\mu))=a^{T}\Sigma e_{i}-a^{T}\mathbb{E}(X)e_{i}^{T}\mu+a^{T}\mathbb{E}(X)e_{i}^{T}\mu.$$

By definition of an eigen vector $\Sigma e_{i}=\lambda_{i} e_{i}$ and $\mathbb{E}(X)=\mu$:

$$ \begin{aligned} \text{Cov}(a^{T}X,e_{i}^{T}(X-\mu))&=a^{T}\lambda_{i} e_{i}-a^{T}\mu e_{i}^{T}\mu+a^{T} \mu e_{i}^{T}\mu \\ &=a^{T}\lambda_{i} e_{i}. \\ \end{aligned} $$