4
$\begingroup$

I understand Regression analysis relies on the following assumptions about the residuals:

  • Normally Distributed (normal plot of residuals)
  • Be independent of each other (random and data must be time ordered)
  • Have a constant variance

Do these same assumptions apply to both Non-Linear regression and linear regression?

$\endgroup$
0

3 Answers 3

1
$\begingroup$

Those assumptions don’t even necessarily apply to linear regression.

The assumptions you list are important in order for the OLS estimator to have the nice properties we want it to have, but that’s just one estimator. A generalized least squares estimator, for instance, could be perfectly reasonable for autocorrelated errors.

Consequently, it is not a given that such assumptions will be important to nonlinear regression models. It will depend on your model and how you want to estimate your model.

$\endgroup$
0
$\begingroup$

There is also the hypothesis of homoskedasticity in the case of linear regression (ie: all residuals must have the same variance)

When jumping into the non-linear regression world, different methods will have different assumptions, so there is no definite answer. I would say, however, that independence of residuals is always a desirable property unless you are working with a model that tries to especifically handle that (time series, for instance)

$\endgroup$
3
  • 1
    $\begingroup$ the OP did say "have a constant variance". And I think the hypothesis is that the residuals are homos[ck]edastic (heteroskedasticity means the variance of the residuals is not constant) $\endgroup$
    – Ben Bolker
    Mar 25, 2019 at 14:00
  • $\begingroup$ It did not say that before the edit. You are right, however, about the confusion between homo and heteroskedasticity $\endgroup$
    – David
    Mar 25, 2019 at 14:11
  • $\begingroup$ So, when dealing with non-linear regression, the assumptions for the residuals are the same? $\endgroup$
    – Aleo
    Mar 26, 2019 at 15:24
0
$\begingroup$

Please be careful that the only assumption on the population error terms we need to get unbiased estimators of the linear regression parameter is that: $$ \mathbb{E}(u|x_1,x_2,\dots,x_n)=0. $$ This means that we assume that unobserved factors are, on average, unrelated to the explanatory variables. Homoskedasticity assumption is not needed (we can calculate HAC standard errors) and is of secondary importance: is only required to obtain the usual variance formulas and to conclude that OLS is best linear unbiased (OLS estimators with the smallest variance). Normality too is not needed: if we can reasonably assume normality, we can obtain the exact sampling distribution of $t$ statistics and $F$ statistics, so that we can carry out exact hypotheses test. Normality assumption can be dropped if we have a reasonably large sample. All the methods of testing and constructing confidence intervals are approximately valid without assuming that the errors are drawn fromm a normal distribution.

Consider now the semi-parametric generalized linear model: $$ Y = m(X,\theta) + \varepsilon, $$ an example may be: $$ Y = \theta_1 \cdot\exp{-\frac{X}{\theta_2}}+\theta_3 \cdot\exp{-\frac{X}{\theta_4}} + \varepsilon $$ no parametric form of $\varepsilon$ is assumed. Here, similarly to the linear model we assume that:

  • $\varepsilon \sim iid$;
  • $\mathbb{E}(\varepsilon|X)=0$
  • $\mathbb{E}(\varepsilon^2)=\sigma^2$

Only for large samples the error terms should be approximately normally distributed. If residuals are heteroskedastic or not normally distributed, than statistical inferences are likely biased. A solution may be to linearize the non-linear equation (for example with logarithms).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.