1
$\begingroup$

I was curious about: Why do we write errors (or disturbance term) $\varepsilon$ as a additive term in a regression model? To elaborate, whether we consider a paramteric or non-paramteric regression model, given by: $$Y_i = m(x_i)+\varepsilon_ i \ \ \ldots \ i=1,\ldots,n$$ where, m(.) is the unknown regression function and the errors are given by $(\varepsilon_1,\ldots,\varepsilon_n)$

Why does this error term come as an additive random variable in a regression model. Couldn't error be written as a multiplicative, like $$Y_i = m(x_i)\varepsilon_ i \ \ \ldots \ i=1,\ldots,n$$

or as an exponent:

$$Y_i = m(x_i)^{\varepsilon_ i} \ \ \ldots \ i=1,\ldots,n$$

Or in any other non-linear form? Is there a particular reason we always think that the true response ($Y_i$) is a sum of some value plus some noise?

$\endgroup$
  • 2
    $\begingroup$ Please see the hits at GLM for examples where the errors are not additive. $\endgroup$ – whuber Dec 19 '18 at 17:36
  • $\begingroup$ You should be selecting the model that is appropriate for your data. The linear model that you have selected is only one model that you could chose from. It happens to have additive errors. Other models exist where this is not the case. As @whuber has stated, GLM models for example, do not have this property. $\endgroup$ – StatsStudent Dec 19 '18 at 18:01
1
$\begingroup$

Multiplicative errors exist, and are sensible. For example, some sensors have a higher error, the higher the domain of the quantity is, they measure. That is why you should always inspect your data, before you do any further processing. If you notice, that the variance scales with the magnitude of your measurement (for example in a time series), it is likely, that you have a multiplicative error. However, if you identified this, it is easy to convert your multiplicative error in a additive error, by simply applying a log transformation to your data: $$ \log{Y_i} = \log{(m(x_i) \epsilon_i)} = \log{m(x_i)} + \log{\epsilon_i} = m_{new}(x_i) + \epsilon_{new, i} $$ Now you can use the model that assume additive error, and convert the results back by simply apply the exponential transformation $\exp(\cdot)$.

Similar could be done with exponential error, however I can't think of a plausible example with exponential error.

Also, as the comments pointed out, there do exist other model, that assume other errors.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.