Currently I am preparing for my exam. Therefore, we have a sample exam with the answers included. Unfortunately, even with the answers it not all makes sense to me. This is the question with the answer provided:

Let $\bf{x}$ be a random vector with expectation $E(\textbf{x})=\boldsymbol{\mu}$ and covariance matrix $Var(\textbf{x})=\bf{\Sigma}$. Let $\textbf{x}=(x_1, \boldsymbol{x_2'})$, where $x_1$ is a scalar and $\boldsymbol{\mu}$ and $\boldsymbol{\Sigma}$ are partitioned conformably. Let $y=x_1-\bf{b'x_2}$$=(1, -\bf{b}')x$ for some conformable nonrandom vector $\textbf{b}$.

The question: Derive $\textbf{b}$ that minimizes the variance of y. Show the covariances between y and the elements of $\bf{x_2}$ are zero for this choice of $\textbf{b}$.

I know that $\boldsymbol{\Sigma}= \begin{bmatrix} \sigma^2 & \boldsymbol{\sigma_{12}} \\ \boldsymbol{\sigma_{21}} & \boldsymbol{\Sigma_{22}} \end{bmatrix}$ (where $\boldsymbol{\sigma_{21}}=\boldsymbol{\sigma_{12}'}$) and that $\boldsymbol{\mu}=(\mu_1, \boldsymbol{\mu_2})'$. Now the variance of y is given as $Var(y)=(1, -\mathbf{b'})\begin{pmatrix} \sigma^2 & \boldsymbol{\sigma_{12}} \\ \boldsymbol{\sigma_{21}} & \boldsymbol{\Sigma_{22}}\end{pmatrix}\begin{pmatrix}1 \\ -\mathbf{b}\end{pmatrix}$. Solving and taking the derivative with respect to $\mathbf{b}$ yields that it is minimized by $\mathbf{b}=\boldsymbol{\Sigma_{22}}^{-1}\boldsymbol{\sigma_{21}}$. Then it is said that the covariances between y and the elements of $\mathbf{x_2}$ are given by:

$E\{y(\mathbf{x_2}-\boldsymbol{\mu_2})\}=(1, -\mathbf{b'})\begin{pmatrix} \sigma^2 & \boldsymbol{\sigma_{12}} \\ \boldsymbol{\sigma_{21}} & \boldsymbol{\Sigma_{22}}\end{pmatrix}\begin{pmatrix}\mathbf{0} \\ \mathbf{I}\end{pmatrix}=\mathbf{0}$

Now I need help here, what is happening? I know that the covariance is given as: $cov(y, \mathbf{x_2})=E\{[y-E(y)][\mathbf{x_2}-E(\mathbf{x_2})]\}$ and $E(\mathbf{x_2})=\boldsymbol{\mu_2}$, but why is $E(y)$ disappearing? And what happens after that first equality sign when it has been set equal to zero?

Hopefully I have clarified the problem enough. Thanks in advance.


Hint (if I understand your question correctly): another formula for covariance is

$$ \text{Cov}(y,\mathbf{x_2}) = E[y\mathbf{x_2}] - E[y](E[\mathbf{x_2}])' $$

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.