What is the relation between two IIN mean zero random variables?

Question

I have trouble proving the following fact in my econometrics homework. The lecturer said that I should merely look at my statistics books, but I cannot seem to find it anywhere! Thus, sorry if it is (too) ignorant a question.

Suppose that random variables $\varepsilon_{1t}$, $\varepsilon_{2t} \sim IIN(0,\Sigma)$ (i.e. identically independently normally distributed with a vector of means equal to $0$ and a variance-covariance matrix $\Sigma$).

How can I then show that $\varepsilon_{1t}=\lambda\varepsilon_{2t}+u_t$, where $\lambda = \frac {\sigma_{12}} {\sigma_{22}}$ and $Var(u_t)=\sigma_{11}-\frac{\sigma_{12}^2}{\sigma_{22}}$ and $u_t$ is a disturbance term? ($\sigma_{ij}$ denotes the corresponding element of the variance-covariance matrix).

All help is greatly appreciated. :)

---

Clarification: I just wanted to add that this question comes from a time-series context. Thus, IIN means that the $\varepsilon$'s are independent over time (i.e. no autocorrelation) and that the distribution does not change. However, there is contemporaneous correlation between the $\varepsilon$'s as they come from a bivariate distribution.

Answer 1

First, I don't think you should say $\varepsilon_{1t}$ and $\varepsilon_{2t}$ are independent and identically distributed, since then $\sigma_{12}=0$ and $\sigma_{11} = \sigma_{22}$.

Second, I'd drop the $t$ subscripts as they seem irrelevant for the question.

Regarding the proof, this sort of thing is a lot easier to deal with via matrices. So look for a book covering linear models at a slightly higher than introductory level. I like Seber, though I've not seen the recent edition. Either at the beginning or in an appendix, there'll be discussion of the multivariate normal distribution, and that if $X$ ~ MVN and $A$ is a fixed matrix, then $A X$ ~ MVN, and that $$\text{var}(A X) = A \text{var}(X) A'$$ which is the key result you need.

Answer 2

By subtraction, you would like to show that when $(\varepsilon_1, \varepsilon_2)$ has a bivariate Normal distribution with covariance $\Sigma$, then $u = \varepsilon_1 - \frac{\sigma_{12}}{\sigma_{22}}\varepsilon_2$ has a Normal distribution. But this is trivial, because $u$, as a linear combination of Normal variates, is necessarily Normal. For the remaining part, compute

$$\eqalign{ Var(u) &= Var(\varepsilon_1 - \frac{\sigma_{12}}{\sigma_{22}}\varepsilon_2) \\ &= Var(\varepsilon_1) - 2\frac{\sigma_{12}}{\sigma_{22}}Covar(\varepsilon_1, \varepsilon_2) + \left(\frac{\sigma_{12}}{\sigma_{22}}\right)^2 Var(\varepsilon_2) \\ &= \sigma_{11} - 2\frac{\sigma_{12}}{\sigma_{22}}\sigma_{12} + \frac{\sigma_{12}^2}{\sigma_{22}^2}\sigma_{22} \\ &= \sigma_{11} - 2\frac{\sigma_{12}^2}{\sigma_{22}} + \frac{\sigma_{12}^2}{\sigma_{22}} \\ &= \sigma_{11} - \frac{\sigma_{12}^2}{\sigma_{22}}. }$$