# What is the relation between two IIN mean zero random variables?

I have trouble proving the following fact in my econometrics homework. The lecturer said that I should merely look at my statistics books, but I cannot seem to find it anywhere! Thus, sorry if it is (too) ignorant a question.

Suppose that random variables $\varepsilon_{1t}$, $\varepsilon_{2t} \sim IIN(0,\Sigma)$ (i.e. identically independently normally distributed with a vector of means equal to $0$ and a variance-covariance matrix $\Sigma$).

How can I then show that $\varepsilon_{1t}=\lambda\varepsilon_{2t}+u_t$, where $\lambda = \frac {\sigma_{12}} {\sigma_{22}}$ and $Var(u_t)=\sigma_{11}-\frac{\sigma_{12}^2}{\sigma_{22}}$ and $u_t$ is a disturbance term? ($\sigma_{ij}$ denotes the corresponding element of the variance-covariance matrix).

All help is greatly appreciated. :)

Clarification: I just wanted to add that this question comes from a time-series context. Thus, IIN means that the $\varepsilon$'s are independent over time (i.e. no autocorrelation) and that the distribution does not change. However, there is contemporaneous correlation between the $\varepsilon$'s as they come from a bivariate distribution.

-
 Please register your account -- you will be able to directly edit your posts. – mbq♦ Sep 6 '11 at 7:37

First, I don't think you should say $\varepsilon_{1t}$ and $\varepsilon_{2t}$ are independent and identically distributed, since then $\sigma_{12}=0$ and $\sigma_{11} = \sigma_{22}$.

Second, I'd drop the $t$ subscripts as they seem irrelevant for the question.

Regarding the proof, this sort of thing is a lot easier to deal with via matrices. So look for a book covering linear models at a slightly higher than introductory level. I like Seber, though I've not seen the recent edition. Either at the beginning or in an appendix, there'll be discussion of the multivariate normal distribution, and that if $X$ ~ MVN and $A$ is a fixed matrix, then $A X$ ~ MVN, and that $$\text{var}(A X) = A \text{var}(X) A'$$ which is the key result you need.

-
For a very thorough treatment of the multivariate normal distribution I can recommend Mardia, Kent and Bibby : amazon.com/dp/0124712525 (expensive to buy, but you may find it in the library). – Bob Durrant Sep 6 '11 at 5:46
Dropping the subscripts is a good idea, but your other prefatory remarks are potentially confusing. Thinking of $(\varepsilon_{1t}, \varepsilon_{2t}) = \mathbf{\varepsilon_t}$ as a vector-valued time series allows us to make sense of the OP's terminology. It reminds us that $\Sigma$ describes the distribution of $\mathbf(\varepsilon_t)$, not the covariance between the values at different times $t$. – whuber Sep 6 '11 at 15:39
@whuber - Oh, I see. It's really the set of $(\varepsilon_1, \varepsilon_2)_t$ that are iid MVN. – Karl Sep 7 '11 at 6:27

By subtraction, you would like to show that when $(\varepsilon_1, \varepsilon_2)$ has a bivariate Normal distribution with covariance $\Sigma$, then $u = \varepsilon_1 - \frac{\sigma_{12}}{\sigma_{22}}\varepsilon_2$ has a Normal distribution. But this is trivial, because $u$, as a linear combination of Normal variates, is necessarily Normal. For the remaining part, compute

\eqalign{ Var(u) &= Var(\varepsilon_1 - \frac{\sigma_{12}}{\sigma_{22}}\varepsilon_2) \\ &= Var(\varepsilon_1) - 2\frac{\sigma_{12}}{\sigma_{22}}Covar(\varepsilon_1, \varepsilon_2) + \left(\frac{\sigma_{12}}{\sigma_{22}}\right)^2 Var(\varepsilon_2) \\ &= \sigma_{11} - 2\frac{\sigma_{12}}{\sigma_{22}}\sigma_{12} + \frac{\sigma_{12}^2}{\sigma_{22}^2}\sigma_{22} \\ &= \sigma_{11} - 2\frac{\sigma_{12}^2}{\sigma_{22}} + \frac{\sigma_{12}^2}{\sigma_{22}} \\ &= \sigma_{11} - \frac{\sigma_{12}^2}{\sigma_{22}}. }

-