3
$\begingroup$

I have trouble proving the following fact in my econometrics homework. The lecturer said that I should merely look at my statistics books, but I cannot seem to find it anywhere! Thus, sorry if it is (too) ignorant a question.

Suppose that random variables $\varepsilon_{1t}$, $\varepsilon_{2t} \sim IIN(0,\Sigma)$ (i.e. identically independently normally distributed with a vector of means equal to $0$ and a variance-covariance matrix $\Sigma$).

How can I then show that $\varepsilon_{1t}=\lambda\varepsilon_{2t}+u_t$, where $\lambda = \frac {\sigma_{12}} {\sigma_{22}}$ and $Var(u_t)=\sigma_{11}-\frac{\sigma_{12}^2}{\sigma_{22}}$ and $u_t$ is a disturbance term? ($\sigma_{ij}$ denotes the corresponding element of the variance-covariance matrix).

All help is greatly appreciated. :)


Clarification: I just wanted to add that this question comes from a time-series context. Thus, IIN means that the $\varepsilon$'s are independent over time (i.e. no autocorrelation) and that the distribution does not change. However, there is contemporaneous correlation between the $\varepsilon$'s as they come from a bivariate distribution.

$\endgroup$
  • $\begingroup$ Please register your account -- you will be able to directly edit your posts. $\endgroup$ – user88 Sep 6 '11 at 7:37
3
$\begingroup$

First, I don't think you should say $\varepsilon_{1t}$ and $\varepsilon_{2t}$ are independent and identically distributed, since then $\sigma_{12}=0$ and $\sigma_{11} = \sigma_{22}$.

Second, I'd drop the $t$ subscripts as they seem irrelevant for the question.

Regarding the proof, this sort of thing is a lot easier to deal with via matrices. So look for a book covering linear models at a slightly higher than introductory level. I like Seber, though I've not seen the recent edition. Either at the beginning or in an appendix, there'll be discussion of the multivariate normal distribution, and that if $X$ ~ MVN and $A$ is a fixed matrix, then $A X$ ~ MVN, and that $$\text{var}(A X) = A \text{var}(X) A'$$ which is the key result you need.

$\endgroup$
  • $\begingroup$ For a very thorough treatment of the multivariate normal distribution I can recommend Mardia, Kent and Bibby : amazon.com/dp/0124712525 (expensive to buy, but you may find it in the library). $\endgroup$ – Bob Durrant Sep 6 '11 at 5:46
  • $\begingroup$ Dropping the subscripts is a good idea, but your other prefatory remarks are potentially confusing. Thinking of $(\varepsilon_{1t}, \varepsilon_{2t}) = \mathbf{\varepsilon_t}$ as a vector-valued time series allows us to make sense of the OP's terminology. It reminds us that $\Sigma$ describes the distribution of $\mathbf(\varepsilon_t)$, not the covariance between the values at different times $t$. $\endgroup$ – whuber Sep 6 '11 at 15:39
  • 1
    $\begingroup$ @whuber - Oh, I see. It's really the set of $(\varepsilon_1, \varepsilon_2)_t$ that are iid MVN. $\endgroup$ – Karl Sep 7 '11 at 6:27
3
$\begingroup$

By subtraction, you would like to show that when $(\varepsilon_1, \varepsilon_2)$ has a bivariate Normal distribution with covariance $\Sigma$, then $u = \varepsilon_1 - \frac{\sigma_{12}}{\sigma_{22}}\varepsilon_2$ has a Normal distribution. But this is trivial, because $u$, as a linear combination of Normal variates, is necessarily Normal. For the remaining part, compute

$$\eqalign{ Var(u) &= Var(\varepsilon_1 - \frac{\sigma_{12}}{\sigma_{22}}\varepsilon_2) \\ &= Var(\varepsilon_1) - 2\frac{\sigma_{12}}{\sigma_{22}}Covar(\varepsilon_1, \varepsilon_2) + \left(\frac{\sigma_{12}}{\sigma_{22}}\right)^2 Var(\varepsilon_2) \\ &= \sigma_{11} - 2\frac{\sigma_{12}}{\sigma_{22}}\sigma_{12} + \frac{\sigma_{12}^2}{\sigma_{22}^2}\sigma_{22} \\ &= \sigma_{11} - 2\frac{\sigma_{12}^2}{\sigma_{22}} + \frac{\sigma_{12}^2}{\sigma_{22}} \\ &= \sigma_{11} - \frac{\sigma_{12}^2}{\sigma_{22}}. }$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.