Consider the class of linear or quasi-linear models that include an autoregressive term, such as autoregressive distributed lag models, ARIMA models, VAR and VECM models, and so forth. In general the lagged value of the dependent variable will include the error term from the lagged period, and if there is autocorrelation between that error and the current error (there always is in economic data), OLS will give biased results. I'm hoping to estimate several such models on my data with R. The R universe offers many different packages to estimate such models by various methods. Among those I have looked at are
A number of these offer one or more forms of penalization:
ARDLautomated selection of lag order for each independent variable by AIC or user-definedfunction.
dLagMautomated selection of lag order for each independent variable by AIC, BIC, or MASE
BigVARLASSO, group LASSO, elastic nat, and several mor exotic alternatives
DREGARmodel selection by AIC, BIC, and several others.
dseprovides a list of information test statistics for estimated models
forecastoffers automated model selection, primarily by AICc, for numerous models
MTScomputes AIC and BIC for most of the models described, and provides some functions for recommending the lag order based on this.
VARshrinkoffers ridge, Baysian, and other methods.
I don't promise I have cought all the instances in which these packages allow penaization, where they have not used one of the words "ridge", "LASSO", "AIC", "BIC", or "penal*", where "*" means any suffix.
I have concluded, rightly or wrongly, that among the forms of penalization that I understand well enough to be comfortable using, a two-stage proceedure that first selects the variables to be included in the model (mainly for parsimony} and then applies a shrinkage technique to the model so selected (mainly to prevent overfitting) has the nicest properties. This could be one of the versions of relaxed LASSO, or it could use the AIC to select the variables and ridge regressio0n for shrinkage.
R also offers many packages that provide different penalization metheds. I have looked at the packages
relaxo. None of these have package documentation that makes explicit reference to lagged variables, autoregression, or dynamic models. Some of them do state that they are estimated by maximum likelyhood, and while dynamic models can be and usually are estimated by maximum likelyhood (I don't know if this is also true for Baysian versions) I do not know if estimating the penalty by an ML method suffices for the penalization to respect the autoregressibve structure.
That, indeed, is my question, or rather, questions.
- Can I validly use one of the AR/dynamic model packages to estimate my base equation, and then simply hand the equation so estimated to the package that provides my penalization method of choice?
- Noting that about half of the dynamic model packages I have looked at provide for some form of penalization, primarily for variable, model, or lag-length selection, suppose I limit myself to these, doing both initial estimation and variable selection in the same package. Can I then properly use one of the packages that does penalization to estimate just the shrinkage/L1 part of the penalization? Does it matter if the penalty is estimated by maximum likelyhood or some other way?
bigtimealso offers regularized estimation of univariate and multivariate time series models. $\endgroup$