4
$\begingroup$

I am trying to figure out the 'standard' way of handling multiplicative error in a linear model, i.e. my model reads:

$$ Y_i = (ax_i + b)\varepsilon_i , \quad \varepsilon_i\sim\mathcal{N}(1, \sigma^2) $$

How do I fit this (in R)? My idea was to use a log transform on both sides:

$$ Z_i = \log(Y_i) = \log(ax_i + b) + \log(\varepsilon_i) $$

The additive errors are now lognormally distributed, but how do I fit this? I would say this is a GLM with link function $g=\exp$ and lognormal errors, but R does not seem to know such a model. Am I missing something? This seems to be an extremely 'standard' model...

Edit: I obviously mixed something up here ;) $\varepsilon_i$ should be lognormally distributed, then it is valid to take logs. But still, the link function would be the exponential - or not? So I tried

glm(z ~ x, data = data.frame(x, z = log(y)), family = gaussian(link = 'exp'))

but the link function does not exist...

$\endgroup$
6
  • 1
    $\begingroup$ How do you want to treat log of negative errors? $\endgroup$
    – Michael M
    Commented Jul 12, 2014 at 12:22
  • $\begingroup$ Good point, somebody was sleeping awake ;) So, there must be a canonical way of treating a model with relative errors - how do you do that? $\endgroup$
    – user51964
    Commented Jul 12, 2014 at 14:27
  • $\begingroup$ You're using a Gaussian error with mean 1 because you want it to be a ratio? You can hope that a small enough variance would keep it from going negative, but you really need something like a Gamma distribution for that. Unfortunately, I don't know enough to venture farther. $\endgroup$
    – Wayne
    Commented Jul 12, 2014 at 15:50
  • $\begingroup$ Exactly, I would say a log-Normal distribution for the multiplicative error would be naturl, as the model in logspace has Normal errors then (keep it simple ;) ) The question now realy is, how do I fit the log-Normal model with R? Am I mixing something up with my link function? $\endgroup$
    – user51964
    Commented Jul 12, 2014 at 16:19
  • 1
    $\begingroup$ Could you instead change the original formulation to: $Y_i=(ax_i+b)(e^{\epsilon_i})$, where $\epsilon=N(0,\sigma^2)$? Or $Y_i=(ax_i+b)(1+\epsilon_i)$? $\endgroup$
    – Wayne
    Commented Jul 12, 2014 at 16:34

1 Answer 1

7
$\begingroup$

This is a standard question on econometrics comprehensive exams. I don't think I've ever seen it used in practice, though. Here is the standard answer.

First, usually you would want to assume $E\{\epsilon_i|X\}=1$ and $V\{\epsilon_i|X\}=\sigma^2$. This is as close as you can get to the usual regression modelling assumptions here. Thinking about the mean of $Y$: \begin{align} E\{Y_i|X\} &= E\left\{ \left( ax_i+b\right)\epsilon_i|X\right\}\\ &= \left( ax_i+b\right)E\left\{ \epsilon_i|X\right\}\\ &= \left( ax_i+b\right)1\\ &= \left( ax_i+b\right) \end{align} Now, let's think about the variance of $Y$: \begin{align} V\{Y_i|X\} &= V\left\{ \left( ax_i+b\right)\epsilon_i|X\right\}\\ &= \left( ax_i+b\right)^2V\left\{ \epsilon_i|X\right\}\\ &= \left( ax_i+b\right)^2\sigma^2 \end{align} Hmmmmm. That kind of reminds me of a heteroskedastic regression model, like: \begin{align} Y_i =& ax_i+b+\nu_i\\ &\; E\left\{ \nu_i|X\right\}=0\\ &\; V\left\{ \nu_i|X\right\}=\left( ax_i+b\right)^2\sigma^2 \end{align} If that were the model, then we know that the BLUE is the GLS estimator. Also, we know that OLS would be unbiased, consistent, asymptotically normal. Also, we would know how to calculate the variance of the OLS estimator.

But that model, the linear one with the $\nu$, is not the model we are given. Here is the trick. We make the model with the $\nu$ be the model we are given by forcing it to be: \begin{align} Y_i=&(ax_i+b)\epsilon_i\\ =&ax_i+b + \left[(ax_i+b)\epsilon_i - ax_i-b \right] \end{align}

So, just call the thing in square brackets $\nu_i$. It's easy to verify now that $\nu_i$, the thing in square brackets, conditional on $x$, has mean zero and variance $\left( ax_i+b\right)^2\sigma^2$. So, this multiplicative errors model is just a cleverly disguised linear model with heteroskedasticity.

To estimate this model, you would just run OLS and use heteroskedasticity-robust standard errors.

$\endgroup$
3
  • $\begingroup$ Hey, cool - tanks for that very informative reply! So now I am just a little bit confused about the proper way of fitting this... Using least squares we tacitly assume that the $\nu_i$ are Gaussians again - fair enough. So for the OLS estimate to work, I need to know the variances in advance for the weights, do I not? However they depend on the fitted values here, so I cannot just use weighted least squares in R. So I am missing the 'canonical' way of finding a,b and $\sigma^2$ again. Is it just minimizing $$\sum_i \frac{(Y_i - ax_i - b)^2}{(ax_i + b)^2}$$ Is that identifiable? $\endgroup$
    – user51964
    Commented Jul 13, 2014 at 10:14
  • 1
    $\begingroup$ @user51964 Sorry for the slow reply. I have been vacationing. No, you would not minimize that quantity. Usually, you would run OLS first. This gives unbiased, consistent, asymptotically normal estimators even in the presence of heteroskedasticity. Nowadays, most economists at least would stop there. We would just use heteroskedasticity-robust (or Huberized, or Huber-White) standard errors along with the OLS estimates. You could, however, use the OLS estimates to construct weights and then use feasible generalized least squares. $\endgroup$
    – Bill
    Commented Jul 23, 2014 at 13:08
  • 1
    $\begingroup$ @user51964 The FGLS estimator would be biased (because we don't know the correct weights, we are using estimated ones). OTOH it is consistent, asymptotically normal, and asymptotically efficient (i.e. equivalent to maximum likelihood). Another approach would be to use maximum likelihood---but the objective function would be more complicated than the one you have written down. You have forgotten the $1/\sqrt{2 \pi \sigma^2}$ on the front of the normal density---the $a$ and $b$ also appear there, as well as inside the $\exp{}$. $\endgroup$
    – Bill
    Commented Jul 23, 2014 at 13:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.