Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Are the confidence and prediction bands around a non-linear regression supposed to be symmetrical around the regression line? Meaning they do not take on the hour-glass shape as in the case of the bands for linear regression. Why is that?
Here is the model in question:
$$
F(x) = \left(\frac{A-D}{1 + \left(\frac x C\right)^B}\right) + D
$$
Here is the figure:
and here's the equation:
Confidence and prediction bands should be expected to typically get wider near the ends - and for the same reason that they always do so in ordinary regression; generally the parameter uncertainty leads to wider intervals near the ends than in the middle
You can see this by simulation easily enough, either by simulating data from a given model, or by simulating from the sampling distribution of the parameter vector.
The usual (approximately correct) calculations done for nonlinear regression involve taking a local linear approximation (this is given in Harvey's answer), but even without those we can get some notion of what's going on.
However, doing the actual calculations is nontrivial and it may be that programs might take a shortcut in calculation which ignores that effect. It's also possible that for some data and some models the effect is relatively small and hard to see. Indeed with prediction intervals, especially with large variance but lots of data it can sometimes be hard to see the curve in ordinary linear regression - they can look almost straight, and it's relatively easy to discern deviation from straightness.
Here's an example of how hard it can be to see just with a confidence interval for the mean (prediction intervals can be far harder to see because their relative variation is so much less). Here's some data and a nonlinear least squares fit, with a confidence interval for the population mean (in this case generated from the sampling distribution since I know the true model, but something very similar could be done by asymptotic approximation or by bootstrapping):
The purple bounds look almost parallel to the blue predictions... but they aren't. Here's the standard error of the sampling distribution of those mean predictions:
which clearly isn't constant.
Edit:
Those "sp" expressions you just posted come straight from the prediction interval for linear regression!
Y-hat +/- sp(Y-hat)
$\endgroup$
– Serge
Feb 5 '14 at 1:07
The mathematics of computing confidence and prediction bands of curves fit by nonlinear regression are explained in this Cross-Validated page. It shows that the bands are not always/usually symmetrical.
And here is an explanation with more words and less math:
First, let's define G|x, which is the gradient of the parameters at a particular value of X and using all the best-fit values of the parameters. The result is a vector, with one element per parameter. For each parameter, it is defined as dY/dP, where Y is the Y value of the curve given the particular value of X and all the best-fit parameter values, and P is one of the parameters.)
G'|x is that gradient vector transposed, so it is a column rather than a row of values. Cov is the covariance matrix (inversed Hessian from last iteration). It is a square matrix with the number of rows and columns equal to the number of parameters. Each item in the matrix is the covariance between two parameters. We use Cov to refer to the normalized covariance matrix, where each value is between -1 and 1.
Now compute
c = G'|x * Cov * G|x.
The result is a single number for any value of X.
The confidence and prediction bands are centered on the best fit curve, and extend above and below the curve an equal amount.
The confidence bands extend above and below the curve by:
= sqrt(c)*sqrt(SS/DF)*CriticalT(Confidence%, DF)
The prediction bands extend a further distance above and below the curve, equal to:
= sqrt(c+1)*sqrt(SS/DF)*CriticalT(Confidence%,DF)
In both these equations, the value of c (defined above) depends on the value of X, so the confidence and prediction bands are not a constant distance from the curve. The value of SS is the sum-of-squares for the fit, and DF is the number of degrees of freedom (number of data points minus number of parameters). CriticalT is a constant from the t distribution based on the confidence level you want (traditionally 95%) and the number of degrees of freedom. For 95% limits, and a fairly large df, this value is close to 1.96. If DF is small, this value is higher.
