# Independent variables prep in multiple linear regression

I have 3 independent variables A, B, C and want to run a multiple linear regression to predict Y. After studying the correlations between:

1) A, Y
2) B, Y
3) A/B, Y
4) (A-B)/(A+B), Y

It turns out that 4) has the highest correlation than all other cases by at least > 0.10. Both 3) and 4) make sense to me as variables A and B are complementary: that is, they represent the items bought in store A versus store B and there are only 2 stores in this problem.

Now, in a simple linear regression I have little doubt that the higher the correlation of the single independent variable the better the fit. But in a multiple linear regression, does it make sense to select the independent variables by looking at the formulas or ratios that shows the highest correlations against Y? In this case using 4) in the regression over 1), 2) or 3) because it has the highest correlation.

• I'm no sure that I understand what you mean by refining. I'm also a little confused of how to interpret $\frac{A-B}{A+B}$. Perhaps having standard deviation at the bottom would be more intuitive (although with only two stores it probably will be the same)... – Max Gordon Dec 6 '11 at 7:00
• Do you really want just to predict Y, or do you want to explain the mechanism by which one or more variables explain Y? If the former, then there's no reason not to use 4). If the latter, you'll have some work to do to make it clear to your audience just how 4) is helpful. It's not always the case that the most useful solution is the one with the highest r-squared. – rolando2 Dec 6 '11 at 13:05
• I want to get the best r-squared. All four cases are sound, but I want to understand if the highest correlated variables produce the highest r-squared in a multiple regression, or if, more in general, analyzing each independent variable correlation to Y is a good way to prepare the best fit model. – Robert Kubrick Dec 6 '11 at 13:57
• Also I am aware that there are ways to select/discard the independent based on anova and the t-test, after the regression is run. I would like to understand if the general idea of increasing the correlation against Y like in my example is a sound method to increase r-squared. – Robert Kubrick Dec 6 '11 at 15:37
• Robert, R-squared is meaningless when you are re-expressing variables. For some discussion of R-squared, see stats.stackexchange.com/questions/13314/…. You should prefer to obtain a model which has a good fit overall and acceptably small residuals. – whuber Dec 6 '11 at 20:22

2. You can (accidentally or arbitrarily) make the correlation arbitrarily close to $\pm 1$ by means of a transformation of the dependent values that creates a single extreme outlier. In the following example $A$ (blue) and $B$ (red) are normally distributed--but always positive--and $C=A+B$ plus normally distributed error, except for a single outlying value at $(A,B,C)=(1, 1/16,20)$. Despite this strong relationship between $C$ and untransformed values of $A$ and $B$, the correlation of $A$ and $C$ is -0.2 (notice the sign is wrong!), the correlation of $B$ and $C$ is -0.25 (wrong sign again), and the correlation of $(A-B)/(A+B)$ and $C$ is +.45 (much stronger than either of the other correlations).