0
$\begingroup$

Consider the class of linear or quasi-linear models that include an autoregressive term, such as autoregressive distributed lag models, ARIMA models, VAR and VECM models, and so forth. In general the lagged value of the dependent variable will include the error term from the lagged period, and if there is autocorrelation between that error and the current error (there always is in economic data), OLS will give biased results. I'm hoping to estimate several such models on my data with R. The R universe offers many different packages to estimate such models by various methods. Among those I have looked at are ARDL, BigVAR, dLagM, DREGAR, dse, dyn, dynlm, forecast, MTS, tsDyn, and VARshrink.

A number of these offer one or more forms of penalization:

  • ARDL automated selection of lag order for each independent variable by AIC or user-definedfunction.
  • dLagM automated selection of lag order for each independent variable by AIC, BIC, or MASE
  • BigVAR LASSO, group LASSO, elastic nat, and several mor exotic alternatives
  • DREGAR model selection by AIC, BIC, and several others.
  • dse provides a list of information test statistics for estimated models
  • forecast offers automated model selection, primarily by AICc, for numerous models
  • MTS computes AIC and BIC for most of the models described, and provides some functions for recommending the lag order based on this.
  • VARshrink offers ridge, Baysian, and other methods.

I don't promise I have cought all the instances in which these packages allow penaization, where they have not used one of the words "ridge", "LASSO", "AIC", "BIC", or "penal*", where "*" means any suffix.

I have concluded, rightly or wrongly, that among the forms of penalization that I understand well enough to be comfortable using, a two-stage proceedure that first selects the variables to be included in the model (mainly for parsimony} and then applies a shrinkage technique to the model so selected (mainly to prevent overfitting) has the nicest properties. This could be one of the versions of relaxed LASSO, or it could use the AIC to select the variables and ridge regressio0n for shrinkage.

R also offers many packages that provide different penalization metheds. I have looked at the packages CDLasso, DLASSO, extlasso, glmnet, lmridge, penalized, and relaxo. None of these have package documentation that makes explicit reference to lagged variables, autoregression, or dynamic models. Some of them do state that they are estimated by maximum likelyhood, and while dynamic models can be and usually are estimated by maximum likelyhood (I don't know if this is also true for Baysian versions) I do not know if estimating the penalty by an ML method suffices for the penalization to respect the autoregressibve structure.

That, indeed, is my question, or rather, questions.

  1. Can I validly use one of the AR/dynamic model packages to estimate my base equation, and then simply hand the equation so estimated to the package that provides my penalization method of choice?
  2. Noting that about half of the dynamic model packages I have looked at provide for some form of penalization, primarily for variable, model, or lag-length selection, suppose I limit myself to these, doing both initial estimation and variable selection in the same package. Can I then properly use one of the packages that does penalization to estimate just the shrinkage/L1 part of the penalization? Does it matter if the penalty is estimated by maximum likelyhood or some other way?
Improve this question
$\endgroup$
5
  • $\begingroup$ Package bigtime also offers regularized estimation of univariate and multivariate time series models. $\endgroup$
    – Richard Hardy
    Feb 18, 2021 at 18:45
  • $\begingroup$ Great that you have provided an extensive overview of packages. However, I find your questions a little hard to follow. What exactly do you mean by 1.? What exactly do you mean by 2.? If things are technically possible, what exactly is your concern (what do you mean by validly)? $\endgroup$
    – Richard Hardy
    Feb 18, 2021 at 18:54
  • $\begingroup$ Dear Richard-- I often seem to find that I don't really understand my question until after I understand the answer. But I'm trying to do is someting like relaxed LASSO on models with distributed lags & autocorrelation, lagged dependent variables, or both. As I read the literature, using LASSO or AICc for variable & lag selection & ridge regression for shrinkage provides much of the simplification of LASSO while retaining the better predictions of ridge. But I am afraid that either L1 or L2 regression, if not integrated with the dynamic model estimation, will yield the biased results of OLS. $\endgroup$
    – andrewH
    Feb 21, 2021 at 20:44
  • $\begingroup$ Coefficient estimates will be biased regardless of what type of regularization you use as long as you use any. But they will be biased without regularization as well due to the time series nature of the data. I wonder if you should care about bias, though; would variance not count equally much, so that you actually care about accurate estimates? What do you mean by dynamic model estimation? $\endgroup$
    – Richard Hardy
    Feb 22, 2021 at 7:18
  • $\begingroup$ I follow many others in using the term "dynamic model" to refer to a wide range of linear time series models (or sometimes GLM) that include lagged dependent variables (LDVs). I intend"dynamic" to modify "models" -- I am not claiming there is a distinctive "dynamic estimation." You are right that I am more concerned with accuracy than bias, but models with LDVs and autocorrelation (most economic time series) have biased estimates of the coefficient on the LDV & I tink this will cause trouble in forecasting. $\endgroup$
    – andrewH
    Feb 2, 2022 at 1:56

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.