I would like to do a CUSUM test for the regression parameters of a nonlinear regression model to analyze possible parameters variations. For linear regression models the CUSUM test is based on the cumulative sums:

$$W_r= \sum^{r}_{t=k-1} \frac{w_t}{s}, r = k + 1,...,n,$$ where $s^2$ is the OLS estimate of the variance of the residuals and $w_t$ is a recursive residuals that can be estimates as follows:

$$w_t = \frac{y_t-x_t'b_{t-1}}{\sqrt{v_t}},$$

where $v_t = (1 + x_{t}'A_{t-1}x_{t})$ (which is the variance of the forecast errors), $A_t = (X_{t}'X_{t})^{-1}$ and $b_{t-1}$ the OLS estimator of $\beta$ using observations $1,...,t-1$. For nonlinear regression:

  • I could replace $s^2$ with the Nonlinear Least Squares (NLS) estimate of the variance of the residuals
  • Replace $x_t'b_{t-1}$ with the nonlinear function $f(x_{t},B_{t-1})$ where $B_{t-1}$ is the NLS estimate of $\beta$ using observations $1,...,t-1$

However, I do not see how I could estimate $v_t$ in the nonlinear regression context. I have also not found literature in which they discuss the use of CUSUM in a nonlinear regression context. How could I use the CUSUM test for a nonlinear regression model?

  • $\begingroup$ I find your question confusing because every element about which it is explicit is a part of linear least squares regression. In what sense is your model "nonlinear," then? $\endgroup$ – whuber Dec 26 '18 at 19:57
  • $\begingroup$ @whuber $f(x_{t},B_{t-1})$ is a nonlinear function. For example: $y_{t} = a ^{x_{t}}$ $\endgroup$ – K. Roelofs Dec 27 '18 at 10:23
  • $\begingroup$ What I don't get is that you seem to be estimating the residual variance in two different ways: implicitly, your definition of $w_t$ suggests the estimate is $v_t,$ and now later you assert it is $f(x_t,B_{t-1}).$ Could you clarify? $\endgroup$ – whuber Dec 27 '18 at 13:43
  • $\begingroup$ $v_t$ is the variance of the forecast errors divided by the variance of the residuals. Such that $w_t \sim N(0,\sigma^2)$ if the model is specified correctly. I have updated the question. Hope it is clear now. $\endgroup$ – K. Roelofs Jan 4 '19 at 13:10

Think of the cusum test like this:

  1. You've got some random process that generates values. In your case, it's some fitting exercise which sets regression parameters, where the regression parameter is the random value.

  2. Your random values have to be independent and identically distributed (i.i.d), and their probability must be given by $p(x[n],\theta)$, where $\theta$ is a parameter vector that your pdf is dependent on. E.g. mean, variance, etc.

  3. Let $X[i]$ be the $i^{th}$ realisation of your random process. According to our pdf the probability of your random process happening as it did is $p(X,\theta)=\prod_{i=0}^np(x[i],\theta)$.

  4. Now the point of the cusum test is to work out whether given some alternative $\theta_a$ for your last $k$ observations, whether $\prod_{i=0}^{n-k}p(x[i],\theta)\prod_{i=n-k+1}^{n}p(x[i],\theta_a)$ is significantly bigger than $p(X,\theta)$. The usual route is to use a log-likelyhood test. The cumulative sum bit ends up being a simplification that falls out in deriving of the closed form of this procedure.

So linear or nonlinear doesn't matter. To use this test all you need is $p(x[i],\theta)$ for your random process, and a way to calculate $p(x[i],\theta_a)$. This latter part is usually done using some form of maximum likelihood estimation. So for example, if your process is i.i.d and you can fit a Guassian distribution to the realisations of your regression parameters then you're in business, and you can follow one of the many cusum guides (e.g. google "Cusum algorith a small review", and check out the pdf).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.