3
$\begingroup$

I would like to ask a (probably very simple) question with regards to multiple linear regression.

I have an experimental formula in the form:

$$ Y \sim \frac{a_0 \cdot X_0}{(a_1 \cdot X1) * (a_2 \cdot X_2)} $$

and I want to set up a linear model which fits the $a_i$ coefficients so as to maximize the goodness of fit, how should I proceed?

I thought to decouple the dependent variables by means of logarithms but I am not sure if this the right way to go. A hint seems to be given in this question (answer by whuber): "[...] marginal changes in the explanatory variables are interpreted in terms of multiplicative (percentage) changes in the dependent variable".

Analytically:

$$ \begin{aligned} Y & \sim e^{\log{\frac{a_0 X_0}{(a_1 X1) * (a_2 X_2)}}} \\ & \sim e^{\log{a_0 X_0} - [\log{a_1 X_1} + \log{a_2 X_2}]} \\ \log{Y} & \sim \log{a_0 X_0} - \log{a_1 X_1} - \log{a_2 X_2} \end{aligned} $$

Not really sure how to work it out. Thanks for any help.

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ Your derivation seems to be based on the presumption that there's a "rule" that $\log(ax) = a \log(x)$. This definitely is not the case! Even after you fix that mistake, your model is not identifiable (consider that if you double $a_1$ and halve $a_2$, or double both $a_0$ and $a_1$, the $Y$ value is unchanged). $\endgroup$
    – Glen_b
    Nov 25, 2012 at 23:35
  • $\begingroup$ @Glen_b : good Lord, you're right. My mind has blown up. So you are saying that there is no way to find the optimal coefficients with mulitple linear regression in this case? $\endgroup$
    – Campa
    Nov 26, 2012 at 22:00
  • 1
    $\begingroup$ If you rewrite the original model as $Y \sim b \frac{X_0}{X_1 X_2}$ where $b = \frac{a_0}{a_1 a_2}$ then you can identify $b$ -- e.g. by the kind of log-transform you suggest, or through a nonlinear least squares model, or through a GLM, or a number of other ways - but you cannot identify the individual $a's$ in your formula. (Is this worth working into an answer? Do you have followup issues?) $\endgroup$
    – Glen_b
    Nov 26, 2012 at 22:05
  • $\begingroup$ @Glen_b : yes, it can work into an answer of course, though I'd be curious to know as well if there is some alternative way to fit the optimal $a_i$ coefficients. $\endgroup$
    – Campa
    Nov 27, 2012 at 18:54
  • $\begingroup$ The way it's set up, there is literally no information that pins them down - recall the various doublings and halvings of $a$'s that don't change $b$. You need something that would identify at least two of them separately. $\endgroup$
    – Glen_b
    Nov 27, 2012 at 22:30

1 Answer 1

Reset to default
3
$\begingroup$

Your model is not identifiable. Consider that if you double $a_1$ and halve $a_2$, or double both $a_0$ and $a_1$ - or many other combinations of increases and decreases, the $Y$ value is unchanged - what could possibly tell you which of an infinite number of possible combinations of $a$'s goes with some particular value of $\frac{Y X_1 X_2}{X_0}$?

If you rewrite the original model as $Y = b \frac{X_0}{X_1 X_2}$ where $b=\frac{a_0}{a_1 a_2}$, then you can identify $b$ -- e.g. by the kind of log-transform you suggest, or through a nonlinear least squares model, or through a GLM, or a number of other ways - but you cannot identify the individual $a$′s in your formula.

The way it's set up, there is literally no information that pins them down individually - recall the various doublings and halvings of $a$'s that don't change $b$.

You need something that would identify at least two of them separately (it's like knowing three things add to 100. It tells you nothing about the size of the individual things in the sum, only a relationship between them - but if you have some other way to pin down two of them, you can figure out the third).

Only the combination I expressed as $b$ is identifiable with your present set-up.

The issue is one of inherent non-identifiability in your model.

http://en.wikipedia.org/wiki/Identifiability

Fisher's comment here:

http://en.wikipedia.org/wiki/Parameter_identification_problem#Estimation_methods_and_disturbances

applies to your situation.

Edit:

Estimation of b by models on the original and the log-scale

You notice in a comment that it's possible to write the model as a linear model in $Y$ as well as $\log Y$.

You need to think about your error term. You can fit a no-constant regression to the model in original form, which would estimate $b$ (I'd suggest also trying the constant just as a check of the model assumption), and you can fit a model to $\log Y$ which has only an intercept (which would estimate $\log b$).

But the assumptions the two models make about the error about the population mean are quite different.

Consider the model fitted to the logs re-expressed as a model in the original $Y$s. In terms of the original $Y$s, the error term is multiplicative and the spread of points about the line is assumed proportional to the mean (that is, the points 'fan out' around an increasing mean/'fan in' around a decreasing mean. In the other model, the error is additive and the spread of points is constant when the mean changes.

You should pay attention to what your model is saying and see whether it corresponds to the data.

You also need to take care if you convert an estimate of $\log b$ back to an estimate of $b$. Means don't carry directly through nonlinear transformations.

Improve this answer
$\endgroup$
3
  • $\begingroup$ thanks for the precious help. One last thing: using $b$ as unique coefficient, I could directly use linear regression as well, with no log-transform, I would say. ? $\endgroup$
    – Campa
    Nov 28, 2012 at 21:06
  • $\begingroup$ comment too long, moving it to an edit in the answer $\endgroup$
    – Glen_b
    Nov 28, 2012 at 21:19
  • $\begingroup$ One suggestion about seeing which model is suited to your situation is to simulate from the fitted model (and exponentiate back for the log-transformed one) and see which looks more like the original data. (There are other, faster ways, but they're less easy to explain in a line or two.) $\endgroup$
    – Glen_b
    Nov 29, 2012 at 21:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.