Multiple linear regression. How to find the most significant independent variables? First question on stackexchange =)
Originally a homework assignment, but my question is one of curiosity. I have one dependent variable (customer satisfaction in % 0->100), and several independent variables (such as age, income), some of which are categorical (for example, level of education). The assignment starts: choose 3-5 independent variables (arbitrarily), of which at least one is categorical . These will be included in a multiple linear model. My question is, how do I evaluate which of these variables have most "explanatory power"? That would be more interesting than choosing them arbitrarily. Searching on google and stack exchange did not give me sufficient information to solve this question.
If I missed any relevant information, please let me know. Thanks.
 A: If the number of regression variables you have is not too large, the best way to do this is to fit all-possible-models (i.e., all possible subsets of the available independent variables).  In this method it is common to create a plot or table that shows some goodness-of-fit statistic as a function of the number of variables used in the regression.  For example, if you have $m$ available independent variables (ignoring categorical for the moment), you could make a table or plot showing the largest $R^2$ value attained by models with $0, 1, 2, ..., m$ variables.  This will allow you to see how much additional explanatory power is added by adding an additional variable, and you can also identify the specific model that has the highest explanatory power using any particular number of variables.
With categorical variables this method is complicated a little bit by the fact that these manifest in multiple independent dummy variables, and it is not usually appropriate to include some of the dummies and exclude others (you generally want to either fully include or exclude the categorical variable).  To deal with this the all-possible-models method generally keeps dummies from the same categorical variable together.
A: You could consider stepwise regression. For example, you can start with none of the variables in your model, and proceed to add them one at a time based on which one reduces the AIC (or some other fit criterion) the most. Once you reach a point at which adding none of the remaining variables is beneficial, you have your final model.
Note that you may end up with different final models depending on which criterion you use and which stepwise direction you choose.
