Do in-sample distributional assumptions (e.g. normally distributed residuals) apply also to out-of-sample predictions? I appreciate that out-of-sample error is important (i.e. comparing how close a model's predictions are to leftout data) but are there any distributional assumptions about the out-of-sample predictions?
For instance, for the model:
\begin{equation}
Y = \alpha + \beta x + N(0, \sigma)
\end{equation}
the residuals are assumed normally-distributed with mean 0 and standard deviation $\sigma$.
If we predict new data from this model ($\hat{y}$) and compare it to some leftout data ($y_l$), should the residuals ($\hat{y} - y_{l}$) also be normally distributed?
 A: It is not necessary that the residuals be normally distributed in linear regression, if you are using the model for prediction.
For prediction, there are some assumptions that are important. If we wite the model as:
$ y =  X\beta + \epsilon$
where $X$ is the model matrix, the main assumptions needed for prediction are:

*

*that $X$ be of full rank


*that the association of  $y$ with $ X\beta$ is linear. When you are using the model to predict responses outside the range of the data on which the model was fitted this is very important, but it is also important for in-sample prediction and prediction of new data within existing bounds.


*the absence of outliers. This isn't really an assumption, but if you have extremely influential observations, they obviously have a large influence on the predictions too. If they are "wrong data" (ie an error in sampling or transciption resulting in, say, the height or weight of an animal being being negative or implausibly small/large) then we would be justified in removing them, but if they are not obviously "wrong", but perrhaps just extreme, then it is a good idea to look at the sensitivity of the predictions to the presence of these.
