Given two models:

  1. $Y_i = \alpha + \beta x_i + Z_i$, $i = 1, 2, 3, \dots , n$

  2. $Y_i = A + B x_i + Z_i$, $i = 1, 2, 3, \dots , n$

where, $\alpha$, $\beta$, $x_i$ are fixed and $A$ , $B$, $Z_i$'s are independent normally distributed random variables with mean zero and variance $\sigma_A^2$, $\sigma_B^2$, $\sigma^2$ respectively.

Is it possible to show that the vector $\mathbf{y} = (Y_1, Y_2, Y_3, \dots, Y_n)'$ is distributed as a multivariate normal?

There is a post demonstrated that the joint distribution of a pair of Gaussian random variables are not necessary a bivariate normal. But how about the models stated above? And under what conditions, can lead the joint distribution of two normal random variables to be a bivariate normal?


The problems are solved now. Thanks @eric_kernfeld.

Since $A$ , $B$, $Z_i$'s are independent, $\mathbf{m} = (A, B, Z_1, Z_2, \dots, Z_n)' \sim N_{n+2}(\mathbf{0}_{n+2}, \Sigma_{n+2})$

$$ \Sigma = \left(\begin{matrix} \sigma_A^2 & 0 & 0 & 0 & \dots & 0 \\ 0 & \sigma_B^2 & 0 & 0 & \dots & 0 \\ 0 & 0 & \sigma^2 & 0 & \dots & 0 \\ 0 & 0 & 0 & \sigma^2 & \dots & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & 0 \\ 0 & 0 & 0 & 0 & \dots & \sigma^2 \end{matrix}\right) $$

Property of multivariate normal distribution:

For any $\mathbf{x}_p \sim N_p(\mathbf{\mu}_p, \Sigma_p)$, constant vector $\mathbf{c}$ of length $k$ and $k \times p$ constant matrix $F$,

$$ \mathbf{c} + F\mathbf{x}_p \sim N_k(\mathbf{c} + F \mathbf{\mu}_p, F \Sigma_p F') \dots \dots (1) $$

We can just find a vector $\mathbf{c}$ and matrix $F$ s.t. $\mathbf{y} = \mathbf{c} + F \mathbf{m}$ and claim $\mathbf{y}$ are also multivariate normal.

Denote $\mathbf{x} = (x_1, x_2, \dots, x_n)'$, $\mathbf{1}_n$ and $\mathbf{0}_n$ are vectors of $1$'s and $0$'s respectively with length $n$.

  • For model 1,

$$ \mathbf{c} = \alpha 1_n + \beta \mathbf{x} \quad , \quad F = \left(\begin{matrix} \mathbf{0}_n & \mathbf{0}_n & I_n \end{matrix}\right) $$

$$ \mathbf{c} + F \mathbf{m} = \left(\begin{matrix} \alpha + \beta x_1 + Z_1 \\ \vdots \\ \alpha + \beta x_n + Z_n \end{matrix}\right) = \mathbf{y} $$

  • For model 2, $$ \mathbf{c} = \mathbf{0}_{n} \quad , \quad F = \left(\begin{matrix} \mathbf{1}_{n} & \mathbf{x} & I_n \end{matrix}\right) $$

$$ \mathbf{c} + F \mathbf{m} = \left(\begin{matrix} A + x_1 B + Z_1 \\ \vdots \\ A + x_n B + Z_n \end{matrix}\right) = \mathbf{y} $$

Since $\mathbf{m}$ follows a multivariate normal distribution, $\mathbf{c}$ and $F$ are both fixed, using property $(1)$ we can claim that $\mathbf{y}$ is also multivariate normal in both model.

  • 1
    $\begingroup$ Independent normal random variables do enjoy a bivariate (more generally, multivariate) normal distribution. The covariance matrix is a diagonal matrix in this case. It is a foul canard (apparently believed ti be true by about 30% of the readership of this site) that independent normal random variables cannot be called jointly normal random variables. Nowhere in the definition is the covariance matrix required to have nonzero off-diagonal entries. In short, there is nothing to show or prove if you want to say that $\mathbf Y$ has a multivariate normal distribution. $\endgroup$ Dec 2 '15 at 3:29
  • 1
    $\begingroup$ In your displayed $\Sigma$ matrix, the entries on the first two lines should be $\sigma_A^2$ and $\sigma_B^2$, not $\sigma_A$ and $\sigma_B$. Also, could you not have picked any other symbol than $B$ for the transformation matrix seeing as how you already used $B$ in $Y_i = A + B x_i + Z_i$?? $\endgroup$ Dec 2 '15 at 19:37
  • $\begingroup$ @DilipSarwate I have corrected the mistakes and change some symbols to make my steps less confusing. $\endgroup$
    – pe-perry
    Dec 3 '15 at 2:33

Your question can be answered using the following theorems:

  1. N independent normal random variables are always jointly normal due to their independence.

  2. Any affine (linear plus a fixed shift) function of a jointly normal random vector is still a multivariate normal. This comes with one caveat: if you multiply by zero or by a matrix $M$ where $M^T$ does not have full column rank, some people will call that normal (or "degenerate Gaussian") and some will not. The density of a degenerate Gaussian cannot be calculated naively: the determinant in the denominator would be zero.

To get started applying these to your second example, my "N" is your n + 2, because it must capture the Z's and A and B.

  • $\begingroup$ I have added the steps. Would you mind checking it? Thanks! $\endgroup$
    – pe-perry
    Dec 2 '15 at 9:20
  • 1
    $\begingroup$ The basic idea is correct, but heed the comments from @DilipSarwate to clean things up. $\endgroup$ Dec 2 '15 at 23:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.