3
$\begingroup$

I have a set of data $(X_i,Y_i)$, $i=1,\ldots,n$ where $X$ and $Y$ are supposed to satisfy the following equation $$ y = \beta_0(1+x^2)^{\beta_1},\quad x>0, \quad\quad (1) $$ I am interested in find $\beta_0$ and $\beta_1$. A natural way is to used a regression method. Hence, taking the logarithm into bothside of $(1)$, I get $$ \log(y) = \log(\beta_0) + \beta_1\log(1+x^2).\quad\quad (2) $$ Here, I meet a problem when I try to apply linear regression to find $\beta_0$ and $\beta_1$: the response variable is in form of $\log(1+x^2)$ instead of $x$ or $x^2$. Thus, my question is: is there a way to do linear regression with this model? In addition, I tried to use Taylor expansion as $$ \log(1+x^2) = x^2 + \sum_{n=1}^{+\infty}\frac{(-1)^nx^{2n+2}}{n+1}. $$ But this formula require $|x|<1$ for the convergence of the series while I have here $x>0$.

Thank you for your comments or suggestions.

$\endgroup$
  • 4
    $\begingroup$ Your original equation doesn't contain any variation. If that's really the case, you don't need regression of any kind. You can just take any two data points (which all must lie exactly on the curve) and solve. $\endgroup$ – Glen_b Jun 13 '16 at 10:06
  • 1
    $\begingroup$ This is for a class exercise? $\endgroup$ – Glen_b Jun 13 '16 at 10:17
  • $\begingroup$ I think the title is a bit misleading. $\endgroup$ – Richard Hardy Jun 17 '16 at 17:25
10
$\begingroup$

Warning: You shouldn't transform an equation until you think about how the noise term comes in; it can be important.

For example, you wrote $y = \beta_0(1+x^2)^{\beta_1}$ but unless all your data lie exactly on the same curve, that equation is untrue -- you $y$ values simply don't equal your transformed $x$-values for any values of $\beta_0$ and $\beta_1$.

Consider these two alternatives:

$\qquad y = \beta_0(1+x^2)^{\beta_1} \times \exp(\eta)\:$ (1)

$\qquad y = \beta_0(1+x^2)^{\beta_1} + \epsilon\:$ (2)

(These allow for the points to be not actually on the "true" curve -- of course these are not an exhaustive list of possibilities)

With equation (1) you might consider taking logs of both sides to fit a linear regression, but with equation (2) that's not the right approach (well you can try but it doesn't work very well$^{[1]}$ - the error term gets tangled up in the equation, and among other effects, the spread about the transformed true curve now becomes a function of the mean).

(Note that - assuming $\eta$ and $\epsilon$ have constant spread, with equation (1) the spread about the true curve will be larger when the mean is larger, while with equation (2) the spread about the true curve will be constant as the mean changes)

The most common way to fit something like equation (2) would be nonlinear least squares.

However, with equation 1, you can do as you did and take logs (I assume natural logs):

$\qquad \log(y) = \log(\beta_0) + \beta_1 \log(1+x^2) + \eta\:$ (1b)

(end of warning)


Your main question seemed to relate to a worry about what to do with $\log(1+x^2)$, but you just supply $\log(1+x^2)$ as a predictor in your regression equation.

If you prefer, you can think of it this way. Let $y^*=\log(y)$ and $x^* = \log(1+x^2)$ (and if you like, let $\alpha_0 = \log(\beta_0)$).

So the equation is now

$\qquad y^* = \alpha_0 + \beta_1 x^* + \eta\:$ (1c)

Now with that you can just go ahead and fit that by supplying $y^*$ and $x^*$ values, (calculated from your original $y$ and $x$ values) to a standard linear regression program.

Here's some plots of example data:

enter image description here

In the left hand side plot ($y$ vs $x$) we can see the increasing spread with mean I mentioned before. The right side plot ($y^*$ vs $x^*$) we see constant spread and linear relationship. (If that LHS plot is not what you expect to happen you may need to rethink your strategy.)

However, even if transformation is reasonable, depending on what the quantity of interest is (what is it you want to find out?), you may not want to simply exponentiate your fits back to get fits on the original scale.

$[1]$ (you might still pretend it's like equation (1) and fit a regression on the log scale in order to get starting values for fitting equation (2).)

$\endgroup$
  • $\begingroup$ regarding the error terms and taking logs, given that log is monotonic, doesn't that imply that minimizing the average log error isn't the same as minimizing the average error? Under what circumstances would that apply? $\endgroup$ – Alex Monras Jun 13 '16 at 17:15
  • 1
    $\begingroup$ @AlexMonras Indeed it isn't the same, that was the entire point of my opening section. Whether one, or the other, or something else makes more sense depends on how the conditional distribution of the response relates to the x-variable, and it also matters whether you're looking at the fitted model as a descriptive statistic or for inference (whether hypothesis testing, CIs or whatever), Equation (1) (at least when accompanied by a few assumptions) shows a case where minimizing sums of squares of error on the log-log scale makes more sense. $\endgroup$ – Glen_b Jun 13 '16 at 17:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.