Could logistic regression be used to detect large errors in least squares regression?

Question

I have the following linear model:

$w^*=\text{arg min}_w\sum_{i=1}^N (Y_i-\sum_{j=1}^M X_{i,j}*w_j)^2$

Let be $T \in N^*$ and $e_i=|Y_i-\sum_{j=1}^M X_{i,j}*w_j|$.

It's possible using logistic regression to predict which errors will be less than $T$ ( $e_i<T$) and greater or equal with $T$ ($e_i \ge T$)?

I will give more in formations for a better understanding:

$N$ represent the number of observations. My data has the following property: the histogram of errors using multiple linear regression has a Laplace distribution. My data come from digital images represented on 8 bits. The $Y_i$ are current pixels and $X_ij$ are neighborhoods pixels. I want to predict which pixels produce errors less than $T$. I want to know what R functions can I use to make a test? T is nit very large, it has the values between 1 and 15 in general.

Answer 1

It would be possible in the case where the original model was heteroskedastic and the heteroskedasticity was related to the covariates. For example,

$y_i \sim \text{N}(x_i^T\beta, \sigma^2x_{i,1}^2)$

where the variance of the $i^{th}$ observation is proportional to the square of the first covariate.

One can imagine, in non-Normal regression situations, similar structures that don't require heteroskedasticity per se. However, the default assumptions of regression models in general involve the errors being independent of the regressors, and the variance of the errors being constant.

On the other hand, if doing, for example, Poisson regression, you're out of the linear model world, but since the variance of the "error" is proportional to the mean, it follows that it is related to the covariates, and such a logistic regression would work - although it would convey no information not already conveyed by the results of the Poisson regression, which fully specify the conditional distributions of the $y_i | x_i$. In the generalized linear / additive model framework, where the likelihood is fully specified, the only way you'll be able to add information to the initial regression by using the logistic regression you suggest is if the initial regression has misspecified (usually by ignoring) the structure of the residuals, e.g., ignored the heteroskedasticity in the linear model presented above.

Nonetheless, your suggestion might reveal something about the structure of the residuals in an exploratory analysis. I suspect, though, that effectively discretizing the residuals by $< T$ or $\ge T$ would usually decrease the information content of them more than it would help clarify the analysis - unless it was an outlier analysis, perhaps.

Answer 2

You are trying to use logistic regression to find a possible structure to your residuals. Residuals should be unstructured. If logistic regression picks up something the model is misspecified. Great.

Important: logistic regression looks for a very specific structure. Your method works only if residuals are broken in a very specific way.

For this sort of detection use non-linear regressions. Kernel or Additive regression on the squared/log residuals are far more useful.
Even better: they will allow you to model the whole variance. This answers your question of where the large errors really are.

Answer 3

I think this would only work if the logistic regression model had access to relevant covariates that are missing from the OLS model. In such a situation (i.e., model misspecification), there could be regions in which the observed response values diverge strongly from the predicted values and the logistic regression model would have the requisite information to detect them.