# Normal Regression Model

Could someone please clarify the part highlighted in red? Why the conditional density? I am having hard time understanding why the statement is about conditional density

I don't understand why saying that $e∼N(0,σ^2)$ implies that $Y∣X$ is Normal. I would like to have a proof .. not just words about the concept

• It's an attempt to apply the definition of "$e\sim N(0,\sigma^2)$". All that's needed is to plug $e = y-(x^\prime\beta+ \alpha)$ into the usual formula for the Normal density. Somehow $\alpha$ was omitted--but isn't it obvious where it belongs? – whuber Sep 18 '17 at 20:14
• I don't understand why saying that $e \sim N( 0, \sigma^2)$ implies that $Y \mid X$. I would like to have a proof .. not just words about the concept – gioxc88 Sep 19 '17 at 11:12
• @HardCore What do you mean "$e \sim N(0, \sigma)$ implies that Y|X"? Are you misreading the conditional operator to mean that $Y$ depends on $X$? $Y,X$ are (in this presentation) jointly observed RVs, so a conditional density exists. Since $e$ is independent of $X$, $Y|X$ is a sequence of constants plus a normal RV, making $Y|X$ normal. – AdamO Sep 19 '17 at 12:36
• In the main post I highlighted a sentence in red. That sentence says the same thing I wrote. You are right !!! I was gonna write $e \sim N(0,\sigma^2)$ implies that $Y \mid X$ is normal. Sorry – gioxc88 Sep 19 '17 at 12:46
• @HardCore But regression model conditions on the observed (i.e. known) X's. Moreover, pleas do not be rude. AdamO is trying to help and you are being rude to him. – Tim Sep 19 '17 at 13:24

Simple linear regression model (let's focus on single predictor case for simplicity) describes relationship of dependent variable $Y$ with independent variable $X$. It tells us what kind of value of $Y$ can we expect when $X=y$, i.e. it models conditional expectation

$$E(Y|X) = \mu = \alpha + \beta X$$

where $Y|X \sim \mathcal{N}(\mu, \sigma^2)$, since it is $e + \mu$, where $e \sim \mathcal{N}(0, \sigma^2)$. It is conditional by definition, because we are interested in the relationship between the variables. If it weren't, we would simply ask "what is the expected value of $Y$?", and we wouldn't care about $X$. Finally, if $Y$ and $X$ were independent, then the model would simplify to

$$E(Y|X) = \alpha + 0 \times X = \alpha$$

where $\alpha$ would be a single-value summary statistic that describes $Y$ and minimizes the squared error, i.e. the mean of $Y$.

• I don't understand why saying that $e∼N(0,σ^2)$ implies that $Y∣X$ is Normal. I would like to have a proof .. not just words about the concept – gioxc88 Sep 19 '17 at 12:19
• @HardCore this comes from the properties of expectations $E(X+c) = E(X) +c$. If you add a constant to the random variable you shift it's mean by the constant. Same if you add random variables $E(X+Y) =E(X)+E(Y)$ and if you sum normally distributed random variables. You are simply shifting the mean from $0$ to $\mu$. – Tim Sep 19 '17 at 12:23
• Yes of course I know that, but first you consider the uncoditional and then the conditional. And again why you all keep saying that $X$ is fixed? $X$ is a random variable that has a joint distribution with $Y$. If I want to study the effect say of the weight Y on the height X, when I draw a sample I observe jointly the 2 RVs $X$ and $Y$. They have a joint density $f(x,y)$. They are both RV. Why $X$ should be fixed in this context? – gioxc88 Sep 19 '17 at 12:26
• Tim I think we are talking about different things. Thank you very much for the aswer though. – gioxc88 Sep 19 '17 at 12:31
• @HardCore when using regression you don't estimate the joint distribution, you simplify this problem to estimating the conditional expectation by conditioning on the observed $X$'s. – Tim Sep 19 '17 at 12:36

In regression modeling, the $X$ (a matrix or vector of covariate values) is considered fixed or given by design. Whether by randomization assignment, or by sampling strategy even simple random sampling, this is often a reflection of how data are collected. If we think of $X$ as a constant, then we can speak simply of the density of the $Y$.

$Y$ then is a bunch of constants plus a normal term, making the $Y$s normally distributed with some mean and standard deviation. This is nice because we can use our observed variables ($Y$ and $X$) to do maximum likelihood and estimate $\beta$ even if we never observe $\epsilon$. If we knew $\epsilon$, regression would be simple arithmetic: subtract residual error, solve a system of equations.

• I can't see how $X$ can be considered fixed as it is a random variable. In fact the conditional expectation $E(Y \mid X)$ itself is a random variable. When you take conditional expectation you are actually conditioning on the $\sigma-field$ generated by $X$, $\sigma(X)$. And you couldn't do it if X were fixed. Every time you draw a sample you observe a different value of X. Of course once the sample has been drawn X is fixed, but in that case everything is fixed. – gioxc88 Sep 19 '17 at 12:22
• @HardCore this is a basic tenant of probability theory: when you condition on something, it is no longer random. In a formula: $E[X|X] = X$. If I throw a coin, it is either heads or tails, the randomness comes about when I speak of repeatedly throwing the coin in the same fashion. – AdamO Sep 26 '17 at 19:58

After a while, I came back to this question that has not received an answer so far.

I was able to give a formal proof of the statement (finally). I will give one in the one dimensional case. The proof with $k$ regressors is simply an extension and does not affect the proof below.

Let's begin stating the hypotesis

1. $Y = \beta X + e$

2. $e \sim N(0, \sigma^2)$

3. $X$ and $e$ are independent

We want to prove that $f_{Y \mid X}(y \mid x)$ is $N(\beta x, \sigma^2)$

We are not making any assumption about the distribution of $X$, and I want to point that $X$ is a RANDOM unlike many people said that $X$ is considered FIXED.

Consequently $X$ has a probability distribution. That being said let's begin with the proof:

$f_{Y \mid X}(y \mid x) = \dfrac{f_{YX}(y,x)}{f_{X}(x)}$

$f_{YX}(y,x)=f_{YX}(\beta X + e,x) = f_{eX}(y- \beta X,x)$

the last equality is true because it is a simple bivariate transformation

The independence of $e$ and $X$ implies that

$f_{eX}(y- \beta X,x) = f_e(y- \beta X)f_X(x)$

Finally, since $e \sim N(0, \sigma^2)$

$f_{Y \mid X}(y \mid x) = f_e(y- \beta X) = \dfrac{1}{2 \sigma \sqrt{(\pi)}}exp{\bigg(\dfrac{1}{2} \Big(\dfrac{y -\beta x}{\sigma} \Big)^2 \bigg)}$

This conclude the proof since the last equality implies that.

$Y \mid X \sim N(\beta x, \sigma^2)$