Distribution of specific column of a random variable after repeated sampling

Question

Suppose $\mathbf X$ is normally distributed with mean $\mu$ and standard deviation $\sigma$. After sampling $n$ samples, we repeat the sampling process $m$ times and the sampling data is stored in an $m\times n$ matrix.

Then, is the data on a specific column (not row) also normally distributed?

My simulation in R shows that it's also normally distributed but I'm not sure if there's a proof for that. Also what about other distributions say chi-Square, exponential, etc.?

More context:

My question actually came from the assumption for the Simple Linear Regression model:

\begin{align} Y=\beta_0 + \beta_1\cdot X + \epsilon \end{align}

where $\epsilon$ is normal random variable (random error). But I also see the same model is written as:

\begin{align} Y_i = \beta_0 + \beta_1\cdot X_i + \epsilon_i \end{align}

where $\epsilon_i$ also normally distributed, and $X_i$ is the specific pair $(X_i, Y_i)$ available in the data set. For which, I suspect that the two models' assumptions are equivalent.

Answer 1

For the time being, let $X = (X_1,\ldots, X_p) \sim N_p(\mu_p, \Sigma)$, $\mu = (\mu_1,\ldots,\mu_p)$ and $a$ a $p\times1$ vector. Then, for $Y = a^\top X$ it holds

$$E(Y) = E(a^\top X) = a^\top \mu$$ $$\text{var}(Y) = a^\top \Sigma a$$

and

\begin{align*} Y \sim N(a^\top \mu, a^\top \Sigma a).\tag{*} \end{align*}

Now take $a = (1,0,\ldots,0)$ and you are done.

Property $(*)$ can be proved by using the characteristic function of $X$. I'll give a proof of this later on.

Answer 2

Suppose $\mathbf X = (X_1, X_2, \ldots, X_n)$ is a random vector with $n$ components is normally distributed with mean $\mu$ and standard deviation $\sigma$.

If you mean that $X_1,\ldots,X_n$ are jointly normal then the definition of joint normality answer your question. And in that case the expected value is an $n$-component vector and the variance is an $n\times n$ matrix.

Joint normality means every linear combination $a_1X_1+\cdots+a_nX_n$ is normally distributed, where $a_1,\ldots,a_n$ are not random.

In particular $X_1$ is the special case where $a_1=1$ and $a_2=\cdots=a_n=0.$

Answer 3

When you make $m$ times a sample of $n$ i.i.d. random variables, then you have generated in all $m\cdot n$ i.i.d. random variables. All these $m\cdot n$ variables are still independent of each other, the computer takes care of that. We could just number them from $X_1$ to $X_{mn}$.

Now you have stored them row by row in a matrix, so row 1 consists of $X_1,..., X_n$, row 2 consists of $X_{n+1},...,X_{n+n}$, row 3 of $X_{2n+1},...,X_{2n+n}$ and so on, and row $m$ of $X_{(m-1)n+1},...,X_{mn}$. If you look at any column, for example the first column $(X_1, X_{n+1},\dots, X_{(m-1)n +1})$, or the $j$-th, $(X_j, X_{n+j},\dots, X_{(m-1)n +j})$, the variables are still independent of each other.

This also holds when the simulated $X_1, \dots, X_{mn}$ have a distribution that is not normal. You only need that they are independent :-)

In many texts about linear regression, one writes $$ \mathbf{Y} = \beta_0 + \beta_1 \mathbf{X} + \epsilon $$ and in reality means $$ \begin{array}{1} Y_1\\ \vdots\\Y_n \end{array} \begin{array}{1} =\\ \ \\= \end{array} \begin{array}{1} \beta_0 +\beta_1 X_1 +\epsilon_1\\\qquad \vdots\\ \beta_0 +\beta_1 X_n +\epsilon_n \end{array}. $$