Suppoese X is the attribute and Y is the response as random variables. We observe (X,Y) jointly as a bi-variate normal variable, then least square estimation of Y as a regression function E(Y|X)
is a linear function $\omega_0 + \omega_1X$.
Now, when we take joint distribution of (X,Y) other than normal distribution (such as uniform), then corresponding least square estimate as a regression function $E(Y|X)$ may not be a linear function.
I have the folllowing questions:
- How can perform linear approximation of the regression function $E(Y|X)$ when (X,Y) is having non-normal joint distribution?
- What would be estimation error of the corresponding linear approximation?