CUSUM test for a Nonlinear Regression Model

I would like to do a CUSUM test for the regression parameters of a nonlinear regression model to analyze possible parameters variations. For linear regression models the CUSUM test is based on the cumulative sums:

$$W_r= \sum^{r}_{t=k-1} \frac{w_t}{s}, r = k + 1,...,n,$$ where $$s^2$$ is the OLS estimate of the variance of the residuals and $$w_t$$ is a recursive residuals that can be estimates as follows:

$$w_t = \frac{y_t-x_t'b_{t-1}}{\sqrt{v_t}},$$

where $$v_t = (1 + x_{t}'A_{t-1}x_{t})$$ (which is the variance of the forecast errors), $$A_t = (X_{t}'X_{t})^{-1}$$ and $$b_{t-1}$$ the OLS estimator of $$\beta$$ using observations $$1,...,t-1$$. For nonlinear regression:

• I could replace $$s^2$$ with the Nonlinear Least Squares (NLS) estimate of the variance of the residuals
• Replace $$x_t'b_{t-1}$$ with the nonlinear function $$f(x_{t},B_{t-1})$$ where $$B_{t-1}$$ is the NLS estimate of $$\beta$$ using observations $$1,...,t-1$$

However, I do not see how I could estimate $$v_t$$ in the nonlinear regression context. I have also not found literature in which they discuss the use of CUSUM in a nonlinear regression context. How could I use the CUSUM test for a nonlinear regression model?

• I find your question confusing because every element about which it is explicit is a part of linear least squares regression. In what sense is your model "nonlinear," then? – whuber Dec 26 '18 at 19:57
• @whuber $f(x_{t},B_{t-1})$ is a nonlinear function. For example: $y_{t} = a ^{x_{t}}$ – K. Roelofs Dec 27 '18 at 10:23
• What I don't get is that you seem to be estimating the residual variance in two different ways: implicitly, your definition of $w_t$ suggests the estimate is $v_t,$ and now later you assert it is $f(x_t,B_{t-1}).$ Could you clarify? – whuber Dec 27 '18 at 13:43
• $v_t$ is the variance of the forecast errors divided by the variance of the residuals. Such that $w_t \sim N(0,\sigma^2)$ if the model is specified correctly. I have updated the question. Hope it is clear now. – K. Roelofs Jan 4 '19 at 13:10

2. Your random values have to be independent and identically distributed (i.i.d), and their probability must be given by $$p(x[n],\theta)$$, where $$\theta$$ is a parameter vector that your pdf is dependent on. E.g. mean, variance, etc.
3. Let $$X[i]$$ be the $$i^{th}$$ realisation of your random process. According to our pdf the probability of your random process happening as it did is $$p(X,\theta)=\prod_{i=0}^np(x[i],\theta)$$.
4. Now the point of the cusum test is to work out whether given some alternative $$\theta_a$$ for your last $$k$$ observations, whether $$\prod_{i=0}^{n-k}p(x[i],\theta)\prod_{i=n-k+1}^{n}p(x[i],\theta_a)$$ is significantly bigger than $$p(X,\theta)$$. The usual route is to use a log-likelyhood test. The cumulative sum bit ends up being a simplification that falls out in deriving of the closed form of this procedure.
So linear or nonlinear doesn't matter. To use this test all you need is $$p(x[i],\theta)$$ for your random process, and a way to calculate $$p(x[i],\theta_a)$$. This latter part is usually done using some form of maximum likelihood estimation. So for example, if your process is i.i.d and you can fit a Guassian distribution to the realisations of your regression parameters then you're in business, and you can follow one of the many cusum guides (e.g. google "Cusum algorith a small review", and check out the pdf).