In the general linear model we assume that different factors have an additive effect on the measured numerical target. For example, in the model we can see that being a male adds, on average, 20 kg to the subjects' weigh and being from France reduce, on average, your weight by 4.5 kg. So the factors (like gender, country, education) have an additive effect on the numerical target (or dependent variable).

I think that in many cases it is more natural to assume that effect is not additive but multiplicative. Are there models that adopt this assumption. Of course one can take a logarithm of the target and then apply the additive model to the "new" target and then, by taking exponent of the sum of the additive effects we get a product of effects (so, a multiplicative model). But, in this approach we have two problems:

  1. Sometimes target is zero. So, we are cannot take a logarithm of it.
  2. By working with the logarithm of the target we do not minimize the square deviation from the original target.

I found related questions but they are not answered yet:

  1. How to model a multiplicative effect from this hypothesis?
  2. How to model a multiplicative effect of a parameter
  • 1
    $\begingroup$ Are your zeros really zeros? Or are they values below certain thresholds? If so, you could still use the log-transformation and consider those observations censored. $\endgroup$ – Björn Nov 27 '15 at 11:28
  • $\begingroup$ @Björn, by "censored" you mean that we do not consider these values in the training. But don't we then get a biased sample and the model will to tend to overestimate values (since it never saw small values). $\endgroup$ – Roman Nov 27 '15 at 12:03
  • 2
    $\begingroup$ What you've asked for is a Gaussian generalized linear model with a log link - see Lognormal Regression?, Choosing between LM and GLM for a log-transformed response variable, & Linear model with log-transformed response vs. generalized linear model with log link. But what you want might well be something else. The presence of exact zeroes would suggest that the assumption of homoskedastic normal errors is inappropriate. $\endgroup$ – Scortchi Nov 27 '15 at 15:00
  • 1
    $\begingroup$ @Roman: that's not what I meant, "censored" = known to be (in this case) below a certain value. The classic example would be values in a blood test, where nothing below a certain concentration can be detected. In the likelihood, one then uses cdf (detection limit) instead of pdf (observed value). $\endgroup$ – Björn Nov 28 '15 at 6:55
  • $\begingroup$ See also How should I model a continuous dependent variable in the $[0,\infty]$ range? for elaboration of @Björn's point. $\endgroup$ – Scortchi Nov 30 '15 at 12:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.