When is there no point in using regression?

Question

Under what circumstances would using regression with two given variables not increase accuracy of prediction?

Answer 1

Considering the OLS case

$$Y_{i}=\alpha+\beta X_{i}$$

One case is when you try to predict using values of $X_{i}$ outside your sample range (extrapolation). Say if your data had $1<X_{i}<10$ in the sample, and you try to predict for when a new value is $X=100$.

In OLS you have a prediction interval for a new value $X_{p}$ of:

$$\hat{Y}_{p}=\overline{Y}+\hat{\beta}(X_{p}-\overline{X})\pm t_{\alpha/2}^{(n-2)}\hat{\sigma}_{Y|X}\sqrt{1+\frac{1}{n}+\frac{X_{p}-\overline{X}}{\sum_{i=1}^{n}(X_{i}-\overline{X})^{2}}}$$

Compare this to a prediction interval without using $X$

$$\hat{Y}_{p}=\overline{Y}\pm t_{\alpha/2}^{(n-1)}\hat{\sigma}_{Y}\sqrt{1+\frac{1}{n}}$$

on comparing the two, if the new covariate $X_{p}$ is far enough away from the mean value $\overline{X}$ the OLS prediction interval will get wider and wider, whereas the unconditional interval remains constant. Further if the $X$ variable is uncorrelated with $Y$ so that $\hat{\beta}=0$, then $\hat{\sigma}_{Y|X}\geq \hat{\sigma}_{Y}$ and the prediction interval is always wider.

Answer 2

When the model assumptions are valid, but the data are not correlated.

When the model assumptions are invalid (e.g. the noise process is heteroskedastic) in which case a regression model may fit the data very well, but provide very poor out-of-sample predictions.

See also the excellent point about extrapolation made by probabilityislogic.