# Performing “all-possible regressions” in R

I am trying to implement all-possible regressions in order to select the best predictors of stock returns from an exhaustive list of potential economic/fundamental variables.

My response variable y (i.e. stock returns) is a panel of 3000 securities (cross-section), each having 384 observations (time-series).

Would anyone please suggest the best way to handle this procedure in R, in the context of panel data? I came across the package leaps, but it addresses the case of y as a response vector rather than a response matrix.

Thank you very much,

• I don't recommend this approach. But to answer your question, you can create a list of all possible of variables using something like expand.grid() to create an matrix for which the columns correspond to variables and the cells correspond to whether the variable is included in that model (there is probably a better way) and then fit each model (row). – Ellis Valentiner Aug 23 '13 at 16:22
• @EllisValentiner Thank you for getting back to me. I might not have been very clear actually. Let's take an example: one of my predictors is Inflation. Inflation has a time-series of values, but cross-sectionally, it is constant for all securities (since it is a macro-variable, and not a firm characteristic). Therefore, I end up with 3000 securities, each having a monthly time series of returns (dependent variable). On the other hand, I have a list of predictors (inflation, oil prices, ...) that each have a time-series of values, but each value is constant for the cross-section. – Mayou Aug 23 '13 at 16:25
• Rather than highly problematic stepwise regression it would be better to use a statistically principled approach as discussed at length in stackexchange. – Frank Harrell Aug 23 '13 at 16:47
• There are many, many threads on this. Think about data reduction (blinded to $Y$) or penalization (L2 - ridge regression, L1 - lasso, combination = elastic net). You might also entertain random forests. – Frank Harrell Aug 23 '13 at 17:39
• "All possible regressions" will not allow you to "select the best possible predictors". If this doesn't make sense / you want to know why, it may help to read my answer here: algorithms-for-automatic-model-selection. There are some approaches that can be used w/ situations like yours. These include penalized methods (ridge regression, LASSO, LARS, elastic net) & cross-validation (try reading some of the highest voted threads under the cross-validation tag). – gung Aug 23 '13 at 17:42

After re-reading your question, I believe you mean to ask about model selection among your candidate predictor variables, and not actually running all possible regressions. Fitting all possible models from a given set of predictors is subject to a high degree of data-mining bias. Since many such sub-models will be highly correlated with each other (because they include almost entirely the same set of factors) you would need to adjust your t-statistics to account for the probability that, among the entire set of correlated models, some models just randomly look successful within the particular sample data you have. Adjusting for so many models would imply that you'd need an unrealistically high t-statistic to have any confidence in coefficients from the model that you finally select.

Some better approaches might be Bayesian linear regression where you specify what prior distribution you think is realistic for the coefficient on each of the predictors, or regularized regression like Lasso or Ridge, where you impose some penalty term for how dense or big the set of estimated coefficients is (e.g. the fitting procedure will try to favor models with fewer terms in a suitable sense).

If you start out from one of these perspectives, then there is less risk in testing a couple of models that you think have strong prior evidence.

But in general, if you simply look at all n-choose-k subsets of factors, for k = 1 through n, then by simple random chance, some model will appear very strong but not due to actual forecast efficacy. You should avoid this.

• Thank you for your detailed answer, very much appreciated. I am in the process of reading about LASSO, its intuition, and implementation in R. I am very new to the field of "model/variable selection" and still confused about the different methods. – Mayou Aug 23 '13 at 17:46

I suggest you to use usual panel data analysis approach [in R this involves the use of plm package). There are three categories of explanatory variables (observed or unobserved) that this approach takes into account. First, the variable which is same across each stock but varies over time (economic fundamentals), second is the variable which varies across each stock but doesn't change over the time (e.g., management style), and third includes the variable which varies over time and also across stock (firm's earnings). If the first two variables are unobserved, they are taken into account by using two-way fixed effects (stock effects and year effects), thus use of panel data avoids omitted variable bias arising from the exclusion of these two categories of variables. So, the only bias that arises from the exclusion is due to the omission of last category of variable. If you are sure that your model includes all that belonging to last category, then there is no omitted variable bias. The significance of these variables indicate that they might be important in influencing the stock returns. That being said, which approach to use depends on the purpose of your research.

• Thank you very much for your response. plm package indeed is great for dealing with large datasets. The purpose of my research (I am a practitioner, not an academic), is to develop a stock selection model in the form of an economic factor model. This will support our current stock selection process in place. I have gathered data for 14 possible predictors (based on economic intuition and some backtesting procedures). However, I need to reduce the dimensionality of my variable space, and select only predictors with a "non-zero" premium. – Mayou Aug 23 '13 at 17:29
• In other words, I need to select the variables that help predict the stock returns, and for which the risk is rewarded by the market. Which "saine" statistical procedure would you recommend in this case? – Mayou Aug 23 '13 at 17:30
• @FrankHarrell I have tried to apply the Fama-MacBeth two-pass regression to discover the predictors that have a statistically significant premium. However, the method is very "questionable" (due to errors-in-variables problem) and leads to very weak results. Any suggestions for an alternative procedure for variable selection? Thank you. – Mayou Aug 23 '13 at 17:32
• I think you can use factor analysis or principle component analysis for the selection of variables. – Metrics Aug 23 '13 at 17:32
• You are right indeed. But when I refer to "Economic factor model", this does not mean that the model only incorporates economic factors. The variables that I am testing fall indeed under several categories: fundamental, technical, economic and behavioral. – Mayou Aug 23 '13 at 18:16