How can I identify the appropriate variables for a prediction model using linear regression?

Question

I need to create a predictive model for the pricing of Airbnb using a linear regression. The data set contains 34 variables and I do not know which of them are suitable. I have already divided the data set into a training and a test data set.

As far as I know, it is not reasonable to use all variables in a multiple regression and then select the ones with the highest significance. So how can I identify the appropriate variables? Should I use the variables with the highest correlation?

Any input would help me a lot, thankyou in advance

Answer 1

Generally:

1. Look at your data! Make relevant plots for your question of interest; ie, compare your response against each of the potential predictors, and potentially if the need arises, look at combinations of features. Use said plots to infer which features might be valuable, and come up with a list of features that tend to be important.
2. Once you have a subset of features that you think look valuable, come up with an objective criterion for judging model fit. This could be AIC, chi-squared test, F test, BIC, etc depending on the complexity of the model and the factors you determine are important (ie, if you want fewer features, that will help determine AIC vs BIC, for instance).
3. Construct models in an objective fashion; ie, forwards selection, backwards selection, etc using the features you identified as valuable, and then determine which model is best on the basis of your objective criterion. Make sure the numbers you are getting are still against a null model; ie, a model with no features. Most modern scientific programming languages have built-in packages that will both of these steps for you.
4. Now, do model inference; ie, look at feature $p$-values, etc.

Do NOT construct your model by starting with inference and picking the one with the "best" feature $p$-values (ie, the model that exhibits best the effect you are interested in), as this defeats the purpose of a statistical or probabilistic model.

Answer 2

These other points should be checked as well:

• there should be no multicollinearity among your explanatory variables. To test for that, you can use Variance Inflation Factor.
• models are usually easier to interpret when continuous variables have been scaled.
• after selecting the appropriate explanatory variables, you can start your modelling approach as @Eric mentioned in his points 2 and 3.

Once you have your minimal model, you still have to check that it is valid:

• inspect the diagnostic plots of your model
• as your data may include serial correlation, you should also test for it with the Durbin-Watson test.

Finally, your approach with a training and a test dataset is good and here is a tutorial on how to implement a linear regression with this logic.