# Lasso for time series - Independence assumptions violated?

I know it is not uncommon to use LASSO for time series. But as LASSO is actually only linear regression or a GLM with a constraint, what about the assumption of independent observations these models have? I don't think it is justified to assume the observations in a time series to be independent, or am I wrong?

• This might not be what you were looking for, but see articles on Adaptive LASSO, i.e., Medeiros, Mendes (2015) "l1-Regularization of High-Dimensional Time- Series Models with Flexible Innovations", where they show that it is able to achieve consistent variable selection and parameter estimation even under ARCH errors.
– runr
Commented Apr 3, 2019 at 13:04

Are you confusing model and estimator here? The linear model with homoskedastic errors as you describe here $$\mathbf{Y} = \mathbf{X}\boldsymbol{\beta} + \boldsymbol{\varepsilon},\;\boldsymbol{\varepsilon}\sim\text{N}_n(\mathbf{0}, \sigma^2\mathbf{I}),$$ is a separate concept from the estimator used to estimate $$\boldsymbol{\beta}$$ from the data $$\{(Y_i, X_i)\}_{i=1}^n,$$ such as the OLS/maximum likelihood estimate $$\boldsymbol{\hat\beta}_{OLS} = (\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T\mathbf{Y},$$ or the LASSO estimate $$\boldsymbol{\hat\beta}_{LASSO} = \underset{\boldsymbol{\beta}}{\text{argmin}}\; \sum_{i=1}^n(Y_i - \mathbf{X}_i\boldsymbol{\beta})^2 + \lambda\sum_{j=1}^p\vert\beta_j\vert.$$ These estimators do not rely on the assumption of independent observations, the OLS estimator is still the maximum likelihood estimate even if $$\boldsymbol{\varepsilon}\sim\text{N}_n(\mathbf{0}, \Sigma),$$ where $$\Sigma$$ has nonzero elements that are not on the diagonal (so that the observations are not independent), it just makes it harder to compute the standard errors of the parameter estimates. For the LASSO estimate, we cannot really compute standard errors anyway, so any assumption of dependence between the observations is inconsequential.
• Thanks for your answer! What I still don't get, the estimator $\beta$ is used to estimate the coefficients of linear regression. And for linear regression I have to assume that my $Y_i$ are independent (what is not fulfilled for time series). So how can Lasso be applied to time series as the use of Lasso is (as I understood) equivalent to the use of linear regression (or GLM) but having a constraint on the coefficients as you defined.. Commented Apr 3, 2019 at 12:34
• You do not need to assume that the $Y_i$'s are independent. Commented Apr 3, 2019 at 12:36
• The estimator (lasso, OLS, etc) is different from the model. To estimate the parameters by using a $L_1-$ penalised regression estimator(lasso) is in no way related to any assumption of independent observations. The assumption of independence in a linear regression model is not crucial, and the parameter estimates(of the effects) are identical with or without it. Commented Apr 3, 2019 at 12:42
• $\boldsymbol{\beta}\sim\text{N}_n(\mathbf{0}, \Sigma),$ ?
• @Nutle An n-dimensional random vector whose joint distribution is Gaussian with mean zero and covariance matrix $\Sigma$. Commented Apr 3, 2019 at 13:04