2
$\begingroup$

Given two models:

  1. $Y_i = \alpha + \beta x_i + Z_i$, $i = 1, 2, 3, \dots , n$

  2. $Y_i = A + B x_i + Z_i$, $i = 1, 2, 3, \dots , n$

where, $\alpha$, $\beta$, $x_i$ are fixed and $A$ , $B$, $Z_i$'s are independent normally distributed random variables with mean zero and variance $\sigma_A^2$, $\sigma_B^2$, $\sigma^2$ respectively.

Is it possible to show that the vector $\mathbf{y} = (Y_1, Y_2, Y_3, \dots, Y_n)'$ is distributed as a multivariate normal?

There is a post demonstrated that the joint distribution of a pair of Gaussian random variables are not necessary a bivariate normal. But how about the models stated above? And under what conditions, can lead the joint distribution of two normal random variables to be a bivariate normal?

Thanks!


The problems are solved now. Thanks @eric_kernfeld.

Since $A$ , $B$, $Z_i$'s are independent, $\mathbf{m} = (A, B, Z_1, Z_2, \dots, Z_n)' \sim N_{n+2}(\mathbf{0}_{n+2}, \Sigma_{n+2})$

$$ \Sigma = \left(\begin{matrix} \sigma_A^2 & 0 & 0 & 0 & \dots & 0 \\ 0 & \sigma_B^2 & 0 & 0 & \dots & 0 \\ 0 & 0 & \sigma^2 & 0 & \dots & 0 \\ 0 & 0 & 0 & \sigma^2 & \dots & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & 0 \\ 0 & 0 & 0 & 0 & \dots & \sigma^2 \end{matrix}\right) $$

Property of multivariate normal distribution:

For any $\mathbf{x}_p \sim N_p(\mathbf{\mu}_p, \Sigma_p)$, constant vector $\mathbf{c}$ of length $k$ and $k \times p$ constant matrix $F$,

$$ \mathbf{c} + F\mathbf{x}_p \sim N_k(\mathbf{c} + F \mathbf{\mu}_p, F \Sigma_p F') \dots \dots (1) $$

We can just find a vector $\mathbf{c}$ and matrix $F$ s.t. $\mathbf{y} = \mathbf{c} + F \mathbf{m}$ and claim $\mathbf{y}$ are also multivariate normal.

Denote $\mathbf{x} = (x_1, x_2, \dots, x_n)'$, $\mathbf{1}_n$ and $\mathbf{0}_n$ are vectors of $1$'s and $0$'s respectively with length $n$.

  • For model 1,

$$ \mathbf{c} = \alpha 1_n + \beta \mathbf{x} \quad , \quad F = \left(\begin{matrix} \mathbf{0}_n & \mathbf{0}_n & I_n \end{matrix}\right) $$

$$ \mathbf{c} + F \mathbf{m} = \left(\begin{matrix} \alpha + \beta x_1 + Z_1 \\ \vdots \\ \alpha + \beta x_n + Z_n \end{matrix}\right) = \mathbf{y} $$

  • For model 2, $$ \mathbf{c} = \mathbf{0}_{n} \quad , \quad F = \left(\begin{matrix} \mathbf{1}_{n} & \mathbf{x} & I_n \end{matrix}\right) $$

$$ \mathbf{c} + F \mathbf{m} = \left(\begin{matrix} A + x_1 B + Z_1 \\ \vdots \\ A + x_n B + Z_n \end{matrix}\right) = \mathbf{y} $$

Since $\mathbf{m}$ follows a multivariate normal distribution, $\mathbf{c}$ and $F$ are both fixed, using property $(1)$ we can claim that $\mathbf{y}$ is also multivariate normal in both model.

$\endgroup$
3
  • 1
    $\begingroup$ Independent normal random variables do enjoy a bivariate (more generally, multivariate) normal distribution. The covariance matrix is a diagonal matrix in this case. It is a foul canard (apparently believed ti be true by about 30% of the readership of this site) that independent normal random variables cannot be called jointly normal random variables. Nowhere in the definition is the covariance matrix required to have nonzero off-diagonal entries. In short, there is nothing to show or prove if you want to say that $\mathbf Y$ has a multivariate normal distribution. $\endgroup$ Dec 2, 2015 at 3:29
  • 1
    $\begingroup$ In your displayed $\Sigma$ matrix, the entries on the first two lines should be $\sigma_A^2$ and $\sigma_B^2$, not $\sigma_A$ and $\sigma_B$. Also, could you not have picked any other symbol than $B$ for the transformation matrix seeing as how you already used $B$ in $Y_i = A + B x_i + Z_i$?? $\endgroup$ Dec 2, 2015 at 19:37
  • $\begingroup$ @DilipSarwate I have corrected the mistakes and change some symbols to make my steps less confusing. $\endgroup$
    – pe-perry
    Dec 3, 2015 at 2:33

1 Answer 1

1
$\begingroup$

Your question can be answered using the following theorems:

  1. N independent normal random variables are always jointly normal due to their independence.

  2. Any affine (linear plus a fixed shift) function of a jointly normal random vector is still a multivariate normal. This comes with one caveat: if you multiply by zero or by a matrix $M$ where $M^T$ does not have full column rank, some people will call that normal (or "degenerate Gaussian") and some will not. The density of a degenerate Gaussian cannot be calculated naively: the determinant in the denominator would be zero.

To get started applying these to your second example, my "N" is your n + 2, because it must capture the Z's and A and B.

$\endgroup$
2
  • $\begingroup$ I have added the steps. Would you mind checking it? Thanks! $\endgroup$
    – pe-perry
    Dec 2, 2015 at 9:20
  • 1
    $\begingroup$ The basic idea is correct, but heed the comments from @DilipSarwate to clean things up. $\endgroup$ Dec 2, 2015 at 23:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.