Confidence limits for a nonlinear regression

Question

I've fitted some weighted data to a nonlinear model (namely a Sèrsic light profile), using least squares.

That is, I have fitted a weighted nonlinear least squares model of the form:

$$I_r = I_0 \exp\left[-\left(\frac{r}{\alpha}\right)^{1/n}\right]+\epsilon_r$$

where $I_0$, $α$ and $n$ are parameters.

How do I then find confidence limits for each parameter, i.e. the 1,2,3 sigmas?

Answer 1

I don't have any sample data for your function, but here is an example with a single exponential.

> library(nls2)
> library(MASS)

> x
 [1]  3.04  3.17  3.32  7.02  8.02  8.26 10.04 14.02 15.02 16.07 17.02 21.02 22.02
[14] 23.02 24.02 28.02 29.02 30.02 34.02 35.02 36.02 41.02 42.02 43.02 44.02 46.02
[27] 50.02 51.02 56.02 57.02 58.02 62.02 63.02 64.02 65.02 80.02 91.02
> y
 [1] 478 484 462 507 496 498 446 409 438 409 374 355 345 336 338 320 315 297 306 278
[21] 264 230 253 240 238 196 193 184 196 199 198 172 153 165 157 132  99
> fit <- nls(y ~ a*exp(b*x), start=list(a=500,b=-0.02))
> summary(fit)

Formula: y ~ a * exp(b * x)

Parameters:
    Estimate Std. Error t value Pr(>|t|)    
a  5.364e+02  8.085e+00   66.34   <2e-16 ***
b -1.876e-02  5.729e-04  -32.74   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 19 on 35 degrees of freedom

Number of iterations to convergence: 3 
Achieved convergence tolerance: 1.698e-07

> confint(fit)
Waiting for profiling to be done...
          2.5%        97.5%
a 520.08993543 552.89982422
b  -0.01993429  -0.01760968

Answer 2

You can always fallback on bootstrapped confidence intervals.

Bootstrap resampling:

1. Let $X$ denote your training dataset. Let $n$ denote the number of samples in your training data. Let $k$ denote the number of resampling iterations you want to perform. The more the better, but $k$ should probably be no fewer than $1000$.

2. for $i=1,2,\dots k$, take a random sample $\tilde{X}_i$ (with replacement) of size $n$ from $X$. Train your model and calculate your model paramters. Let $\tilde{\theta}_i$ denote your fitted parameters trained on the $i^{th}$ resampled data set.

3. You can now calculate confidence intervals by determining the quantiles of $\tilde{\theta} = [\tilde{\theta}_1, \tilde{\theta}_2,\dots \tilde{\theta}_k]$. For example, to obtain a $95\%$ confidence interval, calculate the $2.5\%$ and $97.5\%$ quantiles of $\tilde{\theta}$.

For more on bootstrapping, reference chapters seven and eight of The Elements of Statistical Learning (available for free download).