0
$\begingroup$

If my data is non-linear (assume it follows a quadratic function), how should this be handled using regression? Should I run a regression against the polynomial function or attempt to transform the data in to a linearized model using logs? What are the pros and cons of each?

$\endgroup$
  • 2
    $\begingroup$ What you do (and there are several potentially important alternatives you don't mention) depends on (a) what you know or can assume; (b) your distributional model for the situation - not just the mean, but the spread, or indeed the whole distribution; and (c) to some extent, what kinds of questions you may need to address/information you need to obtain. $\endgroup$ – Glen_b Nov 11 '14 at 22:50
3
$\begingroup$

A quadratic function is still a linear model, because the function is linear in parameters:

$y = a + bx + cx^2$

Personally, I would just use this in regular linear regression. Quadratic functions are difficult to linearize. Log-transforming can linearize exponential functions:

$y = a\mathrm{e}^{bx} \quad\rightarrow\quad \log(y) = \log(a) + bx$

and log-log transformations linearize power functions:

$y = ax^b \quad\rightarrow\quad \log(y) = \log(a) + b\log(x)$

The thing about transforms is that they change the nature of the error structure. In general, log transforms make the error structure multiplicative. For example:

$\log(y) = \log(a) + b\log(x) + \epsilon \quad\rightarrow\quad y = ax^b\mathrm{e}^{\epsilon}$

This is why log transforms are called 'variance-stabilizing' transformations. If your error structure is additive (or homogenous/homoscedastic), then transforming actually performs worse than non-linear regression with additive errors.

In summary, be aware of what you're doing to the error structure when you transform, and as long as your model is linear in parameters, I'd just use ordinary least squares.

$\endgroup$
0
$\begingroup$

how should this be handled using regression? If the project requires the understanding of a non-technical business person, I use linear approach, because you can explain easier to your client making histograms, showing how each variable is correlated with the output.

This scenario is always if you are agree with losing some accuracy, in terms of getting more understanding.

Should I run a regression against the polynomial function or attempt to transform the data in to a linearized model using logs? What are the pros and cons of each?

Logs are not always a good option, because it depends on the nature of the variable. Logs are very good whe you have several extreme values. The result of applying them would be a linear relationship.

Sometimes if you split the variable in ranges (i.e. Age: [18-25], [26-50], [50-85]), you exposed better the relationship between input-output variable. This approach is also good if you want to reduce noise.

The final objective is to have a very clean set of variables in order to use in any data mining model.

For further information about the importance and the "how" prepare data, I suggest the book: Data Preparation for Data Mining by Dorian Pyle

If you want to try splitting data into ranges, in the R language: Create binning in R

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.