# Choosing Variables to a Multiple Regression Model [closed]

Right now, I'm trying to use some georeferenced data to predict revenue for some stores.

My data set has 21 obs (Annual Revenue for 21 stores), but I have 306 variables (Population/pop density, area, income groups, purchasing potential, and so on, for 3 different radius: 1km, 3km and 5km). Here is a print of my dataset.

As I don't have a large number of observations, I cannot run a multiple linear regression, so I decided to analyze the correlation matrix of the variables and to perform a principal component analysis and define which variable would be interesting (or maybe a linear combination). In the figure above, everything in red is 1 and in green is >.9.

My intention is to use these variables to increment another predicition model where I have ~ 4000 variables to predict an hourly numbers of item sales, based in dummy variables for days of week, month, hollidays, distance to payment days, etc, for the 21 stores). Also I'll review that model to discard some variables, ~ 4000 maybe is a little too much and brings me a lot of problems.

Here is the R lm summary to the 4000 variables model:

Residual standard error: 498.4 on 19062 degrees of freedom Multiple R-squared: 0.821, Adjusted R-squared: 0.8164 F-statistic: 179.8 on 486 and 19062 DF, p-value: < 2.2e-16

As I'm quite new to statistics models, I'm not sure if I'm doing something -really- wrong, so I wanted to ask if you have any suggestions about choosing variables to my models

Thank you!

## closed as too broad by rolando2, kjetil b halvorsen, Michael Chernick, mdewey, BernhardJun 8 '18 at 11:49

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

• The standard approach to variable selection in linear models is a penalized regression model like a LASSO. – Sycorax Dec 8 '15 at 19:25
• Your implied approach is not based on statistical principles. For one thing you may be hurting an unsupervised learning approach (principal components) by letting it make use of the correlations with $Y$. For your problem I'd look at variable clustering followed by principal components, or consider sparse principal components analysis. More details are in my RMS course notes and book. – Frank Harrell Nov 11 '17 at 17:11