Dealing with hierarchical (panel, multi-level) data and fixed effects in LASSO?

The question pretty much explains itself. When running a Lasso regression on a lot of indexed (say by time and location) explanatory variables, is it best practice to transform all data using a within transformation over first location and then time? For example...

$$y^{new}_{l,t} = (y_{l,t}- \frac1T\sum_{t=1}^{T} y_{l,t} - \frac1N\sum_{l=1}^{N} y_{l,t} + \bar{\bar{y}})$$

• I think most lasso programs do this automatically – bdeonovic Dec 4 '14 at 17:40
• What does the structure of the data look like then? I guess you have to include the Fixed Effects indicators for every level of the panel, right? – wolfsatthedoor Dec 4 '14 at 17:50

1 Answer

First thing to do is to rearrange your data into a standard form. If you've got $n$ samples and $d$ features, that means you want

• An $n \times d$ input matrix $X$, in which each column has a mean of $0$ and variance of $1$. This ensures that LASSO's regularization effect treats each dimension "fairly" when deciding whether to shrink it to zero.
• A length-$n$ vector $y$ of outputs, which has a mean of $0$. This ensures the LASSO model doesn't need to use a constant term.

You'll probably want to encode time and location as dimensions (ie as extra columns in $X$), though without knowing the details of the problem I can't say for sure.

Anyway, if you feed $X$ and $y$ into a LASSO solver, you'll then get back a length-$d$ weights vector $w$, that you can then interpret in terms of time, location, and whatever other explanatory variables you have.

• So you are arguing for a fixed effects approach with dummies? – wolfsatthedoor Dec 14 '14 at 14:25
• Pretty much. If you want to use a different approach, you'll probably need to consider a different kind of model than plain L1-regularized linear regression. e: Well, you could add various basis functions but without knowing exactly what the problem is I can't offer advice on which might be useful. – Andy Jones Dec 14 '14 at 14:29
• I mean what if I want to find out what covariates are relevant (as dictated by Lasso) conditional on including all of the time and fixed effects, which I consider nuisance parameters? – wolfsatthedoor Dec 19 '14 at 3:03
• Why is this better than using a within estimator if I consider the fixed effects nuisance parameters? – wolfsatthedoor Dec 19 '14 at 15:56
• To demean the data along multiple levels. Basically using the "within" transformation. Check it out en.wikipedia.org/wiki/Fixed_effects_model. Then, after the within-transformation, use Lasso. – wolfsatthedoor Dec 19 '14 at 16:44