Methods for finding the $k$ best regression predictors out of $n$ total predictors Suppose we want to find the $k=7$ best predictors out of a total of $30$ predictors in a linear regression, and that we use some metric like mean square error. Obviously fitting ${30\choose7} = 2035800$ combinations is tedious, and I am wondering what fast algorithms or methods exist for something like this. Thanks
 A: 
Methods for finding the k best regression predictors out of n total
predictors

This sound like the best subset selection criteria. It is explained in Elements of Statistical Learning - Hastie Tibshirani and Friedman; Springer (2008).
An efficient algorithm — the leaps and bounds procedure (Furnival and Wilson, 1974)—makes this feasible for $k$ not too large, let me says no more than $40$.
I take this image from the cited book:

"The best-subset curve (red lower boundary in Figure 3.5) is necessarily decreasing, so cannot be used to select the subset size k. The question of how to choose k involves the tradeoff between bias and variance, along with the more subjective desire for parsimony. There are a number of criteria that one may use; typically we choose the smallest model that minimizes an estimate of the expected prediction error." pag 58.
Indeed to find the model with the smallest EPE is usually the main goal of predictive learning. In order to do that several criteria exist. Most of them are explained in this book. The algorithms for stepwise selection suggested in the Tim's comment stay among them.
A: Many types of model have equations that do not necessarily use all the predictors - i.e. they have built-in feature selection. These could be useful for this problem e.g. you could build a random forest model, specifying all 30 predictors using the ranger package in R. This works by training a number of decision trees and then these trees are averaged - the final model will only include the best predictors. Furthermore, once the model is fitted you can randomly shuffle each predictor’s data one at a time - the difference between the MSE (or other evaluation criterion) before and after the shuffling gives a measure of each predictor's importance. This is because important variables will be affected by this random sampling, whereas unimportant predictors will show minor differences. So you could use these importance scores to find the k best predictors.
You could also use a function like dredge from the MuMIn package on a regression model - this automates the process of building all combinations of your predictors. You can then produce a weighted average of the regression coefficients, where the weights are based on the AICc score of each model. This method could be more useful if you're only interested in using linear regression, but I imagine it might take a while for a model with 30 predictors... in my experience the random forest method mentioned above will be much quicker.
