The question pretty much explains itself. When running a Lasso regression on a lot of indexed (say by time and location) explanatory variables, is it best practice to transform all data using a within transformation over first location and then time? For example...

$ y^{new}_{l,t} = (y_{l,t}- \frac1T\sum_{t=1}^{T} y_{l,t} - \frac1N\sum_{l=1}^{N} y_{l,t} + \bar{\bar{y}}) $

  • $\begingroup$ I think most lasso programs do this automatically $\endgroup$ – bdeonovic Dec 4 '14 at 17:40
  • $\begingroup$ What does the structure of the data look like then? I guess you have to include the Fixed Effects indicators for every level of the panel, right? $\endgroup$ – wolfsatthedoor Dec 4 '14 at 17:50

First thing to do is to rearrange your data into a standard form. If you've got $n$ samples and $d$ features, that means you want

  • An $n \times d$ input matrix $X$, in which each column has a mean of $0$ and variance of $1$. This ensures that LASSO's regularization effect treats each dimension "fairly" when deciding whether to shrink it to zero.
  • A length-$n$ vector $y$ of outputs, which has a mean of $0$. This ensures the LASSO model doesn't need to use a constant term.

You'll probably want to encode time and location as dimensions (ie as extra columns in $X$), though without knowing the details of the problem I can't say for sure.

Anyway, if you feed $X$ and $y$ into a LASSO solver, you'll then get back a length-$d$ weights vector $w$, that you can then interpret in terms of time, location, and whatever other explanatory variables you have.

| cite | improve this answer | |
  • 1
    $\begingroup$ So you are arguing for a fixed effects approach with dummies? $\endgroup$ – wolfsatthedoor Dec 14 '14 at 14:25
  • $\begingroup$ Pretty much. If you want to use a different approach, you'll probably need to consider a different kind of model than plain L1-regularized linear regression. e: Well, you could add various basis functions but without knowing exactly what the problem is I can't offer advice on which might be useful. $\endgroup$ – Andy Jones Dec 14 '14 at 14:29
  • 1
    $\begingroup$ I mean what if I want to find out what covariates are relevant (as dictated by Lasso) conditional on including all of the time and fixed effects, which I consider nuisance parameters? $\endgroup$ – wolfsatthedoor Dec 19 '14 at 3:03
  • 1
    $\begingroup$ Why is this better than using a within estimator if I consider the fixed effects nuisance parameters? $\endgroup$ – wolfsatthedoor Dec 19 '14 at 15:56
  • 1
    $\begingroup$ To demean the data along multiple levels. Basically using the "within" transformation. Check it out en.wikipedia.org/wiki/Fixed_effects_model. Then, after the within-transformation, use Lasso. $\endgroup$ – wolfsatthedoor Dec 19 '14 at 16:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.