Time series forecasting with many predictors Suppose I want to forecast a time series $y$ using its own lags and a large set of potential candidate predictors $X$. The model could be specified as follows:
$$y_t = a + \rho \cdot y_{t-1} + \beta \cdot X_{t-1} + e_t$$
The objective is to find methods with good out-of-sample forecast performance. 
So far I have used bayesian  and non-bayesian shrinkage methods (e.g. Bayesian Model Averaging, Ridge, LASSO, Least Angle Regression) as well as principal component analysis.
Which alternative methods could be promising in this context? 
Preferably I would like to implement them in R if packages are available.
 A: You could use any Machine Learning (ML) method suitable for regression (i.e., dealing with numerical outputs) rather than classification. For this, simply create a predictor vector containing the lagged variables $y_{t-1}$ and treat this just as you would any other predictor.
You would need to calculate the forecasts step-by-step so you can always feed the forecast $\hat{y}_t$ as a predictor into the model to predict $\hat{y}_{t+1}$.
You would need to decide on suitable lag orders by, say, using a holdout sample, since ML algorithms won't give you ACF/PACF. (Or you could try to get an idea of good lags by running ACF/PACF plots on the time series itself, disregarding the predictors you have.)
As for specific algorithms, I personally am a big fan of Random Forests, which are implemented in the randomForest package for R. They can deal with nonlinearities, interactions and large numbers of predictors, and they are fast. However, note that most implementations of RFs can inherently not predict outside the historical range. See here for a discussion - the comments point to a couple of implementations using leaf regression, which can extrapolate, and will dampen your trend, which will usually be a good thing.
Other ML algorithms like k-nearest neighbors are similarly inherently unable to extrapolate. If you have a strongly trended series, keep this in mind. You may want to look at differencing your data first.
A: Transfer Function Modeling proposed in Chapter 10 of the Box-Jenkins textbook(Time Series Analysis: Forecasting and Control 4th Edition) discuss using the Cross-Correlation Function to identify which variables are important. They key is to also consider outliers as they will destroy the relationships.  If you ignore the outliers then you might fail.
There is a new package(not free) that uses this methodology.
http://www.autobox.com/cms/index.php/news/219-autobox-for-r
