Residual Analysis assumptions for non-linear regression

Question

I understand Regression analysis relies on the following assumptions about the residuals:

• Normally Distributed (normal plot of residuals)
• Be independent of each other (random and data must be time ordered)
• Have a constant variance

Do these same assumptions apply to both Non-Linear regression and linear regression?

Answer 1

Those assumptions don’t even necessarily apply to linear regression.

The assumptions you list are important in order for the OLS estimator to have the nice properties we want it to have, but that’s just one estimator. A generalized least squares estimator, for instance, could be perfectly reasonable for autocorrelated errors.

Consequently, it is not a given that such assumptions will be important to nonlinear regression models. It will depend on your model and how you want to estimate your model.

Answer 2

There is also the hypothesis of homoskedasticity in the case of linear regression (ie: all residuals must have the same variance)

When jumping into the non-linear regression world, different methods will have different assumptions, so there is no definite answer. I would say, however, that independence of residuals is always a desirable property unless you are working with a model that tries to especifically handle that (time series, for instance)

Answer 3

Please be careful that the only assumption on the population error terms we need to get unbiased estimators of the linear regression parameter is that: $$ \mathbb{E}(u|x_1,x_2,\dots,x_n)=0. $$ This means that we assume that unobserved factors are, on average, unrelated to the explanatory variables. Homoskedasticity assumption is not needed (we can calculate HAC standard errors) and is of secondary importance: is only required to obtain the usual variance formulas and to conclude that OLS is best linear unbiased (OLS estimators with the smallest variance). Normality too is not needed: if we can reasonably assume normality, we can obtain the exact sampling distribution of $t$ statistics and $F$ statistics, so that we can carry out exact hypotheses test. Normality assumption can be dropped if we have a reasonably large sample. All the methods of testing and constructing confidence intervals are approximately valid without assuming that the errors are drawn fromm a normal distribution.

Consider now the semi-parametric generalized linear model: $$ Y = m(X,\theta) + \varepsilon, $$ an example may be: $$ Y = \theta_1 \cdot\exp{-\frac{X}{\theta_2}}+\theta_3 \cdot\exp{-\frac{X}{\theta_4}} + \varepsilon $$ no parametric form of $\varepsilon$ is assumed. Here, similarly to the linear model we assume that:

• $\varepsilon \sim iid$;
• $\mathbb{E}(\varepsilon|X)=0$
• $\mathbb{E}(\varepsilon^2)=\sigma^2$

Only for large samples the error terms should be approximately normally distributed. If residuals are heteroskedastic or not normally distributed, than statistical inferences are likely biased. A solution may be to linearize the non-linear equation (for example with logarithms).