Right now, I'm trying to use some georeferenced data to predict revenue for some stores.
My data set has 21 obs (Annual Revenue for 21 stores), but I have 306 variables (Population/pop density, area, income groups, purchasing potential, and so on, for 3 different radius: 1km, 3km and 5km). Here is a print of my dataset.
As I don't have a large number of observations, I cannot run a multiple linear regression, so I decided to analyze the correlation matrix of the variables and to perform a principal component analysis and define which variable would be interesting (or maybe a linear combination). In the figure above, everything in red is 1 and in green is >.9.
My intention is to use these variables to increment another predicition model where I have ~ 4000 variables to predict an hourly numbers of item sales, based in dummy variables for days of week, month, hollidays, distance to payment days, etc, for the 21 stores). Also I'll review that model to discard some variables, ~ 4000 maybe is a little too much and brings me a lot of problems.
Here is the R lm summary to the 4000 variables model:
Residual standard error: 498.4 on 19062 degrees of freedom Multiple R-squared: 0.821, Adjusted R-squared: 0.8164 F-statistic: 179.8 on 486 and 19062 DF, p-value: < 2.2e-16
As I'm quite new to statistics models, I'm not sure if I'm doing something -really- wrong, so I wanted to ask if you have any suggestions about choosing variables to my models