Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Could someone please clarify the part highlighted in red? Why the conditional density? I am having hard time understanding why the statement is about conditional density
I don't understand why saying that $e∼N(0,σ^2)$ implies that $Y∣X$ is Normal. I would like to have a proof .. not just words about the concept
thank you
Simple linear regression model (let's focus on single predictor case for simplicity) describes relationship of dependent variable $Y$ with independent variable $X$. It tells us what kind of value of $Y$ can we expect when $X=y$, i.e. it models conditional expectation
$$ E(Y|X) = \mu = \alpha + \beta X $$
where $Y|X \sim \mathcal{N}(\mu, \sigma^2)$, since it is $e + \mu$, where $e \sim \mathcal{N}(0, \sigma^2)$. It is conditional by definition, because we are interested in the relationship between the variables. If it weren't, we would simply ask "what is the expected value of $Y$?", and we wouldn't care about $X$. Finally, if $Y$ and $X$ were independent, then the model would simplify to
$$ E(Y|X) = \alpha + 0 \times X = \alpha $$
where $\alpha$ would be a single-value summary statistic that describes $Y$ and minimizes the squared error, i.e. the mean of $Y$.
In regression modeling, the $X$ (a matrix or vector of covariate values) is considered fixed or given by design. Whether by randomization assignment, or by sampling strategy even simple random sampling, this is often a reflection of how data are collected. If we think of $X$ as a constant, then we can speak simply of the density of the $Y$.
$Y$ then is a bunch of constants plus a normal term, making the $Y$s normally distributed with some mean and standard deviation. This is nice because we can use our observed variables ($Y$ and $X$) to do maximum likelihood and estimate $\beta$ even if we never observe $\epsilon$. If we knew $\epsilon$, regression would be simple arithmetic: subtract residual error, solve a system of equations.
After a while, I came back to this question that has not received an answer so far.
I was able to give a formal proof of the statement (finally). I will give one in the one dimensional case. The proof with $k$ regressors is simply an extension and does not affect the proof below.
Let's begin stating the hypotesis
$Y = \beta X + e $
$e \sim N(0, \sigma^2)$
$X$ and $e$ are independent
We want to prove that $ f_{Y \mid X}(y \mid x)$ is $N(\beta x, \sigma^2)$
We are not making any assumption about the distribution of $X$, and I want to point that $X$ is a RANDOM unlike many people said that $X$ is considered FIXED.
Consequently $X$ has a probability distribution. That being said let's begin with the proof:
$f_{Y \mid X}(y \mid x) = \dfrac{f_{YX}(y,x)}{f_{X}(x)}$
$f_{YX}(y,x)=f_{YX}(\beta X + e,x) = f_{eX}(y- \beta X,x)$
the last equality is true because it is a simple bivariate transformation
The independence of $e$ and $X$ implies that
$f_{eX}(y- \beta X,x) = f_e(y- \beta X)f_X(x)$
Finally, since $e \sim N(0, \sigma^2)$
$f_{Y \mid X}(y \mid x) = f_e(y- \beta X) = \dfrac{1}{2 \sigma \sqrt{(\pi)}}exp{\bigg(\dfrac{1}{2} \Big(\dfrac{y -\beta x}{\sigma} \Big)^2 \bigg)} $
This conclude the proof since the last equality implies that.
$ Y \mid X \sim N(\beta x, \sigma^2)$
