1
$\begingroup$

These two figures give the plot of x vs y, and how to transform either or both x or y variables to fit a linear regression. How to interpret the following transformation schemes? Especially, how to interpret the last column “Linear Form” and “Linearizable Function” given the transformation? note: Figure a and b are the first two graphs. c and d are the second two below a and b and so on.

enter image description here

enter image description here

$\endgroup$

1 Answer 1

3
$\begingroup$

Regression via OLS can only be applied if the model equation is linear in the parameters. This means that all parameters $\beta_i$ that are to be estimated need to be either added, subtracted or multiplied to the regressors $X_i$. They cannot be a power or be in a logarithm for example (any non-linear operation). This is so because non-linear parameters cannot be written in matrix form and hence cannot be minimized algebraically yielding a unique solution via minimizing a squared loss (numerical optimizations are possible). The restriction does not hold for the regressors themselves, hence "linear regression" does not mean you can't have for example $X^2$, but that you can't have for example $2^\beta$.

In your table, the "linearizable functions" are functions non-linear in the parameters (note how the $y$s are not linear in the $\beta$s). However, by transforming $y$ using the function in the column "transformation", the function can be made linear in the parameters and thus estimate-able via OLS. The form shown in the last column "linear form" is the form you can do OLS with.


Take for example the first row:

  1. $y = \beta_0 x^{\beta_1}$. Here $\beta_0$ is linear, $\beta_1$ is in the exponent.

  2. Take the log of 1. to get $\log(y)=\log(\beta_0)+\beta_1\log(x)$. Here, $\beta_1$ has been linearized, so no more problems there. However, $\beta_0$ is inside the $\log$ which is not linear. However, when we do the LS regression, we will get $\log(\beta_0)$ as an estimate which we can then simply exponentiate to get $\beta_0$. Since no regressor $x$ is attached to (e.g. multiplied by) $\beta_0$, this is no problem.

  3. Define $\log(y)=y'$ and $log(x)=x'$. Now to do OLS, simply use $\log(y)$ as a response and $\log(x)$ as a regressor (instead of the original $y$ and $x$).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.