# How to "choose" categorical variables which have impact in a regression?

I have a dataset of about 50K samples. I have approximately 90 columns which are all categorical and they're used to predict a price. There's no other continuous value. I'm trying to select "which of those columns" have a greater impact on the final response variable. Here's an example:

Suppose we're trying to estimate salaries of developers. The variables we have are:

• Role: Data Scientist
• Main Language: Python
• IDE Used: PyCharm
• Country: UK
• Years of experience: +5 years

How can you "measure" the impact of these variables on the final salary? How can you decide which variables to use for your regression and which ones to leave aside because they're not "relevant" to the final dependent variable?

Thanks!

• Do you mean you have 90 different predictors (I am not sure what you mean by a column). You can run a linear regression with 90 dummy variables although you really need to try to come up with some theory to limit them to a more manageable numbers. While methods such as stepwise regression exist to suggest what variables to add many are extremely critical of using such automatic approaches to select variables. From painful experience statistical approaches, including regression slope really don't tell you relative impact. Mar 19, 2019 at 22:28

• All-possible-models: This might be computationally infeasible with so many available variables, but it might be possible to implement if you can cut-down with some other first-pass method. In this approach the research fixed a value $$m$$ for the number of parameters in the model and finds $$\min SSE(n)$$ taken over all possible models with that number of parameters. The researcher then plots the function $$n \mapsto \min SSE(n)$$ and looks for an appropriate number of parameters that reduces the residual-sum-of-squares to a desired level. This can be done by formal methods using partial F-tests, but often the researcher will use graphical assessment. (In the latter case the researcher will usually look for a "kink" in the function indicating that the rate of decrease of the -sum-of-squares is rapidly diminishing, so that there is little value in adding more parameters.)