# Shape of confidence and prediction intervals for nonlinear regression

Are the confidence and prediction bands around a non-linear regression supposed to be symmetrical around the regression line? Meaning they do not take on the hour-glass shape as in the case of the bands for linear regression. Why is that?

Here is the model in question:
$$F(x) = \left(\frac{A-D}{1 + \left(\frac x C\right)^B}\right) + D$$ Here is the figure:

and here's the equation:

• Your question is unclear, because you shift from asking if they are "supposed to be symmetrical" in the 1st sentence, to implying that they are not in sentence 2 & asking (presumably) why they aren't in sentence 3. Can you make this more consistent / clear? – gung - Reinstate Monica Feb 4 '14 at 22:15
• OK, let me ask it this way - why are the confidence and prediction bands symmetrical around the regression line when the regression is non-linear but take on an hour-glass shape when it is linear? – Serge Feb 4 '14 at 22:32
• Just some comments, in case they're of any help: it looks like your responses must be non-negative and converge to $0$ (or close to it) at $0$, whereas these bands evidently are erected using a model of independent additive error. That makes them unrealistic, especially at the left. Moreover, the patterns of blue dots suggest the error has strong serial correlation, which also needs to be accounted for in constructing these bands. Although you might not want to cope with this additional complexity in your data to do the fitting, it indicates the bands you have drawn aren't worth much. – whuber Feb 5 '14 at 21:53
• You are right. The band does cross into the negative territory. However, I am not interested in the values of the bands themselves, but rather in the EC50 values corresponding to the band limits. Is there an alternative to constructing the bands this way? – Serge Feb 6 '14 at 0:27
• Yes, but as I intimated they can get complicated. Generalized least squares and time series methods can cope with the serial correlation. Nonlinear transformations of the dependent variable are one tool to handle non-additive error. A more sophisticated tool is a generalized linear model. The choices depend partly on the nature of the dependent variable. BTW, although I'm unsure what you mean by "EC50 values" (it sounds like you are modeling dose-response relationships), anything computed from the illustrated bands will be suspect. – whuber Feb 6 '14 at 16:24

Confidence and prediction bands should be expected to typically get wider near the ends - and for the same reason that they always do so in ordinary regression; generally the parameter uncertainty leads to wider intervals near the ends than in the middle

You can see this by simulation easily enough, either by simulating data from a given model, or by simulating from the sampling distribution of the parameter vector.

The usual (approximately correct) calculations done for nonlinear regression involve taking a local linear approximation (this is given in Harvey's answer), but even without those we can get some notion of what's going on.

However, doing the actual calculations is nontrivial and it may be that programs might take a shortcut in calculation which ignores that effect. It's also possible that for some data and some models the effect is relatively small and hard to see. Indeed with prediction intervals, especially with large variance but lots of data it can sometimes be hard to see the curve in ordinary linear regression - they can look almost straight, and it's relatively easy to discern deviation from straightness.

Here's an example of how hard it can be to see just with a confidence interval for the mean (prediction intervals can be far harder to see because their relative variation is so much less). Here's some data and a nonlinear least squares fit, with a confidence interval for the population mean (in this case generated from the sampling distribution since I know the true model, but something very similar could be done by asymptotic approximation or by bootstrapping):

The purple bounds look almost parallel to the blue predictions... but they aren't. Here's the standard error of the sampling distribution of those mean predictions:

which clearly isn't constant.

Edit:

Those "sp" expressions you just posted come straight from the prediction interval for linear regression!

• are you also saying that the increase in the parameter uncertainty as one moves away from the center should cause the band to widen at the ends even in the case of nonlinear regression, but that it is just not as obvious? Or is there a theoretical reason why this widening does not happen in the case of nonlinear regression? My bands certainly look very symmetrical. – Serge Feb 4 '14 at 22:44
• That widening would happen should be typical, but it will not happen the same way with every nonlinear model and will not be as obvious with every model, and because it's not as easy to do may not be calculated that way by a given program. I don't know how the bands you're looking at have been calculated - I am not a mind reader, and I can't see the code of a program you haven't even mentioned the name of. – Glen_b -Reinstate Monica Feb 4 '14 at 22:47
• @user1505202, this remains a difficult question to answer fully. Can you state what your model is (its functional form)? Can you attach an image of the figure that is perplexing to you? – gung - Reinstate Monica Feb 4 '14 at 22:48
• Thanks. I do have the numbers and they are essentially constant - the difference between the regression line and each prediction limit ranges from 18.21074 in the middle to 18.24877 at the ends. So, a slight widening, but very slight. By the way, @gung, I got the equation that calculates the prediction interval. It is: Y-hat +/- sp(Y-hat) – Serge Feb 5 '14 at 1:07
• That's about the sort of variation you might see with a prediction interval with largish samples. What's sp? – Glen_b -Reinstate Monica Feb 5 '14 at 1:08

The mathematics of computing confidence and prediction bands of curves fit by nonlinear regression are explained in this Cross-Validated page. It shows that the bands are not always/usually symmetrical.

And here is an explanation with more words and less math:

First, let's define G|x, which is the gradient of the parameters at a particular value of X and using all the best-fit values of the parameters. The result is a vector, with one element per parameter. For each parameter, it is defined as dY/dP, where Y is the Y value of the curve given the particular value of X and all the best-fit parameter values, and P is one of the parameters.)

G'|x is that gradient vector transposed, so it is a column rather than a row of values. Cov is the covariance matrix (inversed Hessian from last iteration). It is a square matrix with the number of rows and columns equal to the number of parameters. Each item in the matrix is the covariance between two parameters. We use Cov to refer to the normalized covariance matrix, where each value is between -1 and 1.

Now compute

c = G'|x * Cov * G|x.

The result is a single number for any value of X.

The confidence and prediction bands are centered on the best fit curve, and extend above and below the curve an equal amount.

The confidence bands extend above and below the curve by:

= sqrt(c)*sqrt(SS/DF)*CriticalT(Confidence%, DF)

The prediction bands extend a further distance above and below the curve, equal to:

= sqrt(c+1)*sqrt(SS/DF)*CriticalT(Confidence%,DF)

In both these equations, the value of c (defined above) depends on the value of X, so the confidence and prediction bands are not a constant distance from the curve. The value of SS is the sum-of-squares for the fit, and DF is the number of degrees of freedom (number of data points minus number of parameters). CriticalT is a constant from the t distribution based on the confidence level you want (traditionally 95%) and the number of degrees of freedom. For 95% limits, and a fairly large df, this value is close to 1.96. If DF is small, this value is higher.

• Thanks, Harvey. I am working on getting the gradient of the parameters for my function. Do you by any chance know of a worked example, for I am not clear on how the covariance matrix is obtained either. – Serge Feb 7 '14 at 22:57
• If you use the GraphPad Prism demo, you can fit data to any model you want, and view the covariance matrix (an optional result chosen in the Diagnostics tab) and the confidence or prediction bands (as both numbers and a graph; also choose in Diagnostics tab). That isn't quite a good as a worked example, but at least you can compare the covariance matrix and see if the problem is before or after... – Harvey Motulsky Feb 7 '14 at 23:40
• Two things, though. 1. Prism did give me the Cov matrix. However, it is only one number for the entire dataset. Am I not supposed to get one value per X value? 2. I get the prediction band in the graph but I would like the output to contain the values. Prism does not appear to do that. I am very new to Prism and so I may not have looked everywhere, but I tried! – Serge Feb 10 '14 at 18:53
• 1. The covariance matrix shows the degree to which the parameters are intertwined. So there is one value for every pair of parameters that you ask nonlinear regression to fit. 2. Look on the Range tab to ask Prism to make a table of the XY coordinates of the curve, with plus/minus values for the confidence or prediction bands. 3. For tech support with Prism, email support@graphpad.com Use this forum for statistical questions, not tech support. – Harvey Motulsky Feb 10 '14 at 19:25