I'm working on a regression model with the Hitters data from the ISLR package.
It has ~300 observations and 20 variables. I want to predict a player's salary.
I have major problems of collinearity and multicollinearity.
Initialize data
library('ISLR')
data(Hitters)
attach(Hitters)
Check for multicollinearity
library('mctest')
imcdiag(num_vars(Hitters[-19]),Hitters$Salary)
Examine Output
Call:
imcdiag(x = num_vars(Hitters[-19]), y = Hitters$Salary)
All Individual Multicollinearity Diagnostics Result
VIF TOL Wi Fi Leamer CVIF Klein
AtBat 22.4794 0.0445 353.6936 380.4917 0.2109 -5.2419 1
Hits 30.0835 0.0332 478.9086 515.1937 0.1823 -7.0152 1
HmRun 7.6367 0.1309 109.2844 117.5645 0.3619 -1.7808 1
Runs 15.1175 0.0661 232.4681 250.0814 0.2572 -3.5252 1
RBI 11.6885 0.0856 176.0040 189.3392 0.2925 -2.7256 1
Walks 4.0903 0.2445 50.8876 54.7431 0.4944 -0.9538 1
Years 9.1263 0.1096 133.8123 143.9508 0.3310 -2.1281 1
CAtBat 250.0646 0.0040 4101.2642 4412.0019 0.0632 -58.3124 1
CHits 495.6521 0.0020 8145.2706 8762.4079 0.0449 -115.5807 1
CHmRun 46.2836 0.0216 745.6703 802.1670 0.1470 -10.7928 1
CRuns 158.6813 0.0063 2596.4848 2793.2109 0.0794 -37.0028 1
CRBI 131.2023 0.0076 2143.9984 2306.4413 0.0873 -30.5950 1
CWalks 19.7303 0.0507 308.4253 331.7935 0.2251 -4.6009 1
PutOuts 1.2304 0.8128 3.7936 4.0810 0.9015 -0.2869 0
Assists 2.7002 0.3703 27.9969 30.1181 0.6086 -0.6297 1
Errors 2.1842 0.4578 19.5002 20.9777 0.6766 -0.5093 1
1 --> COLLINEARITY is detected by the test
0 --> COLLINEARITY is not detected by the test
HmRun , Runs , RBI , Years , CAtBat , CHits , CHmRun , CRBI , Assists , Errors , coefficient(s) are non-significant may be due to multicollinearity
R-square of y on all x: 0.5279
View correlation matrix
I see three areas for problematic collinearity. The runs/hits/hmruns/rbi/walks group in the bottom left of the matrix, the years/catbat/chits/so on in the middle of the matrix and the error:assist pair in the top right of the matrix.
I know my options include: drop one of the variables involved, change them to an interaction term.
The problem is that it seems like I'm just guessing. I have no reason to think error better describes what's going on than assist and it is too many potential combinations to try them all.
What's the better alternative for dealing with collinearity?