How to choose the best transformation to achieve linearity?

Question

I want to do multiple linear regression and then to predict new values with little extrapolation. I have my response variable in the range from -2 to +7, and three predictors (the ranges about +10 - +200). The distribution is nearly normal. But the relationship between the response and the predictors is not linear, I see curves on the plots. For example like this: http://cs10418.userapi.com/u17020874/153949434/x_9898cf38.jpg

I would like to apply a transformation to achieve linearity. I tried to transform the response variable by checking different functions and looking at the resulting plots to see a linear relationship between the response and predictors. And I found that there are many functions which can give me visible linear relationship. For example, functions

$t_1=\log(y+2.5)$

$t_2=\frac{1}{\log(y+5)}$

$t_3=\frac{1}{y+5}$

$t_4=\frac{1}{(y+10)^3}$

$t_5=\frac{1}{(y+3)^\frac{1}{3}}$ etc. give the similar results: http://cs10418.userapi.com/u17020874/153949434/x_06f13dbf.jpg

After I am going to back-transform the predicted values (for $t=\frac{1}{(y+10)^3}$ as $y’=\frac{1}{t^\frac{1}{3}}-10$ and so on). The distributions are more or less similar to normal.

How can I choose the best transformation for my data? Is there a quantitative (and not very complicated) way to evaluate linearity? To prove that selected transformation is the best or to find it automatically if possible.

Or the only way is to do the non-linear multiple regression?

Answer 1

This is somewhat of an art, but there are some standard, straightforward things one can always attempt.

The first thing to do is re-express the dependent variable ($y$) to make the residuals normal. That's not really applicable in this example, where the points appear to fall along a smooth nonlinear curve with very little scatter. So we proceed to the next step.

The next thing is to re-express the independent variable ($r$) to linearize the relationship. There is a simple, easy way to do this. Pick three representative points along the curve, preferably at both ends and the middle. From the first figure I read off the ordered pairs $(r,y)$ = $(10,7)$, $(90,0)$, and $(180,-2)$. Without any information other than that $r$ appears always to be positive, a good choice is to explore the Box-Cox transformations $r \to (r^p-1)/p$ for various powers $p$, usually chosen to be multiples of $1/2$ or $1/3$ and typically between $-1$ and $1$. (The limiting value as $p$ approaches $0$ is $\log(r)$.) This transformation will create an approximate linear relationship provided the slope between the first two points equals the slope between the second pair.

For example, the slopes of the untransformed data are $(0-7)/(90-10)$ = -$0.088$ and $(-2-0)/(180-90)$ = $-0.022$. These are quite different: one is about four times the other. Trying $p=-1/2$ gives slopes of $(0-7)/(\frac{90^{-1/2}-1}{-1/2}-\frac{10^{-1/2}-1}{-1/2})$, etc., which work out to $-16.6$ and $-32.4$: now one of them is only twice the other, which is an improvement. Continuing in this fashion (a spreadsheet is convenient), I find that $p \approx 0$ works well: the slopes are now $-7.3$ and $-6.6$, almost the same value. Consequently, you should try a model of the form $y = \alpha + \beta \log(r)$. Then repeat: fit a line, examine the residuals, identify a transformation of $y$ to make them approximately symmetric, and iterate.

John Tukey provides details and many examples in his classic book Exploratory Data Analysis (Addison-Wesley, 1977). He gives similar (but slightly more involved) procedures to identify variance-stabilizing transformations of $y$. One sample dataset he supplies as an exercise concerns century-old data about mercury vapor pressures measured at various temperatures. Following this procedure enables one to rediscover the Clausius-Clapeyron relation; the residuals to the final fit can be interpreted in terms of quantum-mechanical effects occurring at atomic distances!

Answer 2

If your response variable (or rather, what will become the residuals of your response variable) on the original scale has a Normal distribution as you imply, then transforming it to create a linear relationship with the other variables will mean that it no longer is Normal and it will also change the relationship between its variance and mean values. So from that part of your description I think you are better off using non-linear regression than transforming the response. Otherwise, after linear transformation of the response, you will need a more complex error structure (although this can be a matter of judgement and you would need to check, using graphical methods).

Alternatively, investigate transformation of the explanatory variables. As well as straight transformations, you also have the option of adding in quadratic terms.

More generally, transformation is more an art than a science, if there is no existing theory to suggest what you should use as the basis of transformation.