Consider two regression models:

$log(y_i) = \log(x_i)\alpha + \epsilon_i \,\,\,\,\,$ (Model 1),

$log(y_i) = (\frac{x_i}{\overline{x}})\beta + \varepsilon_i \,\,\,\,\,\,\,\,$ (Model 2),

where $\overline{x}$ is the sample average of $x_i$.

Both of these models transform the variable $x_i$, the first with a log, the second by dividing by the sample average.

In short, why aren't $\alpha$ and $\beta$ equal?

I am confused because, it is my understanding that:

$\alpha = \frac{\partial \log(y)}{\partial \log(x)} = \frac{\partial \log(y)}{\partial y}\frac{\partial y}{\partial x} \frac{\partial x}{\partial \log(x)} = \frac{\partial y}{\partial x}\frac{x}{y}$


$\beta = \frac{\partial \log(y)}{\partial (\frac{x_i}{\overline{x}})} = \frac{\partial \log(y)}{\partial y}\frac{\partial y}{\partial x} \frac{\partial x}{\partial (\frac{x_i}{\overline{x}})} = \frac{\partial y}{\partial x}\frac{x}{y}$

However, in simulations, these two regression coefficients do not exactly equal each other. Is there an approximation going on somewhere in my definitions that I am ignoring? Is there some kind of small-sample bias that is relevant in practice that is missed here?

  • 1
    $\begingroup$ You probably need to provide either code from which the simulation can be read or a detailed description of the simulation. Personally I get very similar estimates. If you make a first order Taylor approximation of the log function around $\bar x$ then you get $\log(x) \approx \log(\bar x) + (x-\bar x)/\bar x$ this identity can be used to go from model 1 to model 2. $\log(y) = \lambda + \log(x)\alpha + u$ becomes $\log(y) = \lambda + [\log(\bar x) + (x-\bar x)/\bar x]\alpha + u$ which becomes $\log(y) = (\lambda + \log(\bar x)\alpha - \alpha) + (x/\bar x)\alpha + u$. $\endgroup$ – Jesper for President Jan 14 at 10:46
  • 1
    $\begingroup$ IF you include a constant the OLS estimates of the coefficients of $(x/\bar x)$ and $\log(x)$ should not change that much. $\endgroup$ – Jesper for President Jan 14 at 10:49
  • $\begingroup$ This is very helpful. And I get very similar estimates as well but they are not exact. Is there a way to “undo” the Taylor approximation in order to write $\alpha$ as a function of $\beta$? $\endgroup$ – km5041 Jan 14 at 12:02
  • $\begingroup$ I do not think so. $\endgroup$ – Jesper for President Jan 14 at 12:45

You made a small mistake when taking the derivative of Model 2. $$log(y) = \Bigl(\frac{x_i}{\bar{x}}\Bigr)\beta + e_i$$ If you are differentiating with respect to $x_i$ then $\bar{x}$ and $\beta$ are treated as constants exactly the same way if you were differentiating $x*5*2$, the derivative of which would be just 10. Therefore:

$$\frac{1}{y}\frac{dy}{dx_i}= \frac{\beta}{\bar{x}}$$ $$\beta =\frac{dy}{dx_i}\frac{\bar{x}}{y} $$

And hence the two are not equal. On an intuitive level you can think, that since the original two models apply different transformations to the $x$ variable it is logical that the OLS parameters $\alpha$ in Model 1 and $\beta$ in Model 2 will give different results.

On another important note, if you are trying to estimate the regression coefficient, you are differentiating the wrong functions with respect to the wrong variables. The OLS regression estimates regression coefficients by minimizing the sum of squared residual, and hence the funciton that you need to take the derivative of is: $$\sum_{i=1}^{n}{\hat{u}_i}^2 = \sum_{i=1}^{n}{(y_i - \hat{y}_i)^2} =\sum_{i=1}^{n}{(y_i-\hat{\beta}x_i)^2} $$ You can see the full derivation of the estimation here: https://are.berkeley.edu/courses/EEP118/current/derive_ols.pdf

For a multivariate case where you have many different variables such that $y = \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 ... +e$ see this:

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.