As a partial answer to your question, you can build a residual model (also known as a variance model) to
model the residuals for the original model as a function of the predicted response (for example) as follows (in R notation):
residual.model <-
lm(abs(residuals(original.model)) ~ predict(original.model), ...)
You can then examine the residual model to get a deeper understanding
of the data. From the residual model you can estimate prediction
intervals, for example.
The residuals are usually much more noisy than the original data
used to build the model, and so there will be more uncertainty about the
residual model than the original model.
The above residual model uses lm and thus assumes (at most) a linear relationship between
the absolute residuals and the predicted response, which is often a good enough
approximation to the (unknown) underlying reality---or maybe I should say, a linear residual model is often about the
most complicated residual model you would want to use given the noise in the
residuals.
My
earth
R package builds variance models essentially using the above idea
(but the ideas are fairly universal and apply not just to earth/MARS
models). Some of the background theory can be found in the package vignette
Variance models in earth.
When reading the vignette, mentally substitute the name of your model
instead of "earth" e.g. substitute "lm" for "earth".
Additional references can be found in the above vignette. Especially helpful is Carroll and Ruppert
Transformation and Weighting in Regression.
