I've got data regarding baseball with four independent variables. I'm confused as to how to determine which variable is the most significant predictor. To begin, I ran a multiple regression and focused on the t-values stated in the coefficient's table. However, 3 of the 4 variables are significant according to the p-values. How can I go about resolving this? Should I run 4 separate linear regressions and then compare the F statistics instead?
There are multiple ways to determine the best predictor. One of the most easy way is to first see correlation matrix even before you perform the regression. Generally variable with highest correlation is a good predictor. You can also compare coefficients to select the best predictor (Make sure you have normalized the data before you perform regression and you take absolute value of coefficients) You can also look change in R-squared value. Trying removing each one of them and see which variable causes maximum change.