Calculating the noise on data fitting an exponential decay

Question

I'm trying to calculate the amount of noise in data that fits to an exponential decay function. I'm trying to calculate signal-to-noise at different times of the data. Here is the code for how I create some data:

import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit

time = np.arange(0,100,0.5)
a_perf = [10 * np.exp(-t / 20) for t in time] #perfect decay
noise = np.random.uniform(0, 0.1, size = len(a_perf)) #noise
a_noisey = a_perf + noise

popt, pcov = curve_fit(exp_dec, t, a_noisey)

And that creates the following plot:

The fitting function from popt gives a fit of A = 9.99789709, tau = 20.38314338 and from the covariance matrix, I can calculate the uncertainty on these to be np.sqrt(np.diag(pcov)) = [ 0.01129902, 0.03303147] so A(uncertainty) = 0.01129902, tau(uncertainty) = 0.03303147. I want to be able to calculate how much noise in the data and am unsure how to. I am thinking I should take the residual of the data to the fit and then can use that to calculate noise but am not sure. What should I look into reading and learning in order to do this? Also, here is the time and data that I used:

t = [  0. ,   0.5,   1. ,   1.5,   2. ,   2.5,   3. ,   3.5,   4. ,
         4.5,   5. ,   5.5,   6. ,   6.5,   7. ,   7.5,   8. ,   8.5,
         9. ,   9.5,  10. ,  10.5,  11. ,  11.5,  12. ,  12.5,  13. ,
        13.5,  14. ,  14.5,  15. ,  15.5,  16. ,  16.5,  17. ,  17.5,
        18. ,  18.5,  19. ,  19.5,  20. ,  20.5,  21. ,  21.5,  22. ,
        22.5,  23. ,  23.5,  24. ,  24.5,  25. ,  25.5,  26. ,  26.5,
        27. ,  27.5,  28. ,  28.5,  29. ,  29.5,  30. ,  30.5,  31. ,
        31.5,  32. ,  32.5,  33. ,  33.5,  34. ,  34.5,  35. ,  35.5,
        36. ,  36.5,  37. ,  37.5,  38. ,  38.5,  39. ,  39.5,  40. ,
        40.5,  41. ,  41.5,  42. ,  42.5,  43. ,  43.5,  44. ,  44.5,
        45. ,  45.5,  46. ,  46.5,  47. ,  47.5,  48. ,  48.5,  49. ,
        49.5,  50. ,  50.5,  51. ,  51.5,  52. ,  52.5,  53. ,  53.5,
        54. ,  54.5,  55. ,  55.5,  56. ,  56.5,  57. ,  57.5,  58. ,
        58.5,  59. ,  59.5,  60. ,  60.5,  61. ,  61.5,  62. ,  62.5,
        63. ,  63.5,  64. ,  64.5,  65. ,  65.5,  66. ,  66.5,  67. ,
        67.5,  68. ,  68.5,  69. ,  69.5,  70. ,  70.5,  71. ,  71.5,
        72. ,  72.5,  73. ,  73.5,  74. ,  74.5,  75. ,  75.5,  76. ,
        76.5,  77. ,  77.5,  78. ,  78.5,  79. ,  79.5,  80. ,  80.5,
        81. ,  81.5,  82. ,  82.5,  83. ,  83.5,  84. ,  84.5,  85. ,
        85.5,  86. ,  86.5,  87. ,  87.5,  88. ,  88.5,  89. ,  89.5,
        90. ,  90.5,  91. ,  91.5,  92. ,  92.5,  93. ,  93.5,  94. ,
        94.5,  95. ,  95.5,  96. ,  96.5,  97. ,  97.5,  98. ,  98.5,
        99. ,  99.5]
a_noisey = [ 10.00414965,   9.777553  ,   9.57845509,   9.29113291,
         9.06226379,   8.83453263,   8.64911463,   8.4927241 ,
         8.2779904 ,   7.99508969,   7.85743129,   7.6750355 ,
         7.42404898,   7.29205261,   7.12600942,   6.91543572,
         6.72753756,   6.55638198,   6.43174929,   6.23847798,
         6.11346604,   5.93217494,   5.85095577,   5.66467134,
         5.5320938 ,   5.38631447,   5.3054303 ,   5.14940904,
         5.04740488,   4.87192248,   4.7642406 ,   4.60721089,
         4.59067802,   4.47526479,   4.33380734,   4.17734358,
         4.07906655,   3.99889553,   3.92719317,   3.8567559 ,
         3.70643031,   3.60956366,   3.51366982,   3.51296059,
         3.33728311,   3.3198497 ,   3.18260438,   3.12182624,
         3.08859242,   3.00087496,   2.91762832,   2.86985901,
         2.82218524,   2.68530464,   2.66299052,   2.59453132,
         2.49324011,   2.41862409,   2.35825358,   2.35364415,
         2.28694101,   2.23673583,   2.15770272,   2.16298967,
         2.05223805,   2.0236233 ,   1.97591928,   1.92216013,
         1.84718088,   1.81560256,   1.74378234,   1.75494344,
         1.72297737,   1.69287212,   1.62646367,   1.58838596,
         1.50501319,   1.47763476,   1.4460889 ,   1.46462637,
         1.39315949,   1.35207521,   1.38566313,   1.32946105,
         1.2435213 ,   1.2123759 ,   1.24181945,   1.22834474,
         1.19494081,   1.1625428 ,   1.06909585,   1.1065621 ,
         1.08191302,   1.04533487,   0.97446128,   0.93780378,
         0.94012114,   0.89451648,   0.94629686,   0.86253227,
         0.85615738,   0.84072623,   0.79885072,   0.83963017,
         0.83697936,   0.73579196,   0.80645952,   0.71801807,
         0.67906034,   0.74949171,   0.71336805,   0.71064774,
         0.65441762,   0.65122552,   0.66737537,   0.58636957,
         0.56729976,   0.6269349 ,   0.53145966,   0.5217697 ,
         0.55924964,   0.52825405,   0.54606971,   0.48722635,
         0.54011467,   0.44215787,   0.45895263,   0.45492897,
         0.42823244,   0.43705311,   0.43705522,   0.40363142,
         0.41967354,   0.4389169 ,   0.43961019,   0.41993618,
         0.37755745,   0.40698051,   0.31893059,   0.40781878,
         0.37356552,   0.35559183,   0.34292504,   0.30989571,
         0.30591261,   0.32000779,   0.27505043,   0.25995785,
         0.27835992,   0.29608078,   0.24421458,   0.30819081,
         0.26557427,   0.31739917,   0.31202117,   0.26215456,
         0.28922166,   0.27470341,   0.19343073,   0.2155477 ,
         0.19635959,   0.19089341,   0.18945418,   0.21217587,
         0.22164332,   0.22563163,   0.20713921,   0.19306336,
         0.23809653,   0.1674818 ,   0.16258075,   0.1392952 ,
         0.16471956,   0.18549053,   0.19152122,   0.14993843,
         0.20023447,   0.17982779,   0.17254161,   0.15358746,
         0.19557888,   0.16402165,   0.18264406,   0.10741189,
         0.1495215 ,   0.12644875,   0.17131193,   0.16255527,
         0.1506267 ,   0.09215507,   0.11163815,   0.13741619,
         0.09566767,   0.112785  ,   0.14180809,   0.10286331,
         0.15904264,   0.16072477,   0.08766294,   0.08993143]

Answer 1

The usual measure of the size of noise about a nonlinear least squares regression fit is the standard error of the residuals, which is the square root of the variance estimate $$s^2=\hat{\sigma}^2=\frac{1}{n-p}\sum_i (y_i-\hat{y}_i)^2\,.$$

It is an estimate of the population standard deviation of the noise term ($\epsilon$) in the nonlinear regression model $y=f(x;\theta)+\epsilon$.

Here $n$ is the number of observations, $p$ the number of free parameters used to define the fitted model, $y_i$ is the $i$-th response value and $\hat{y}_i$ is the $i$-th fitted value.

[See, for example equation 9.35 and the immediately preceding equations here]

Many programs or functions for fitting nonlinear regression produce a value for $s$ in their output.

For example, here's R output for one of the examples in its help on nonlinear regression:

summary(fm1DNase1)

Formula: density ~ SSlogis(log(conc), Asym, xmid, scal)

Parameters:
     Estimate Std. Error t value Pr(>|t|)    
Asym  2.34518    0.07815   30.01 2.17e-13 
xmid  1.48309    0.08135   18.23 1.22e-10 
scal  1.04146    0.03227   32.27 8.51e-14 
---

Residual standard error: 0.01919 on 13 degrees of freedom

Number of iterations to convergence: 0 
Achieved convergence tolerance: 3.281e-06

The value in the third-last printed line, for "Residual standard error" is just such an estimate.

[Unfortunately the online curve_fit documentation is constantly timing out right now, so I can't look up how to get it from scipy's curve_fit for you.]

---

Would I use the same formula if I were to linearize my data and then just do a least-squares fit?

Note that the two assumptions (that the noise variance is constant on both the original scale and on the log scale) are never consistent with each other - if one constant variance assumption were true, the other could not be.

Your model for the way the noise is spread around the model should actually describe the way the noise behaves.

If the noise is relative for the exponential model (i.e. average spread about the curve is constant in percentage terms, so that typical spread at $x$ divided by expected value at $x$ is about the same at each $x$) then it will be constant in absolute terms on the log scale for the linear model.

On the other hand, if it's constant (in absolute terms) on the original exponential model then the spread will be smaller at larger $x$-values and larger at smaller $x$-values on the log scale.

You can get a poor fit (sometimes quite poor) if you assume constant variance when it's far from correct. Standard errors of parameters will be wrong, and prediction intervals (for new points) will be next to useless.

If you applied the formula for $s$ to a situation where the spread of the noise term wasn't almost constant, the value of $s$ would be estimating a sort of weighted average of those changing spreads, but would really only describe the spread at one place.

That said, if the 'true' model has constant spread on the log scale, then the usual estimate for it uses the same formula.

It's also possible to calculate (at least approximately) the way the spread should go (as a function of the expected value of the model at $x$) after you transform from a constant spread (in what ever direction).

If the size of the error is tiny or the values never get near 0, it may not make much appreciable difference -- so