Lasso/ridge regressions with binary factors

I have started working on analyses of SNP's in the human genome, and I want to apply as many models as I can. But I don't know of any analogues to LASSO / ridge regression (or maybe even elastic net) with binary / trinary factors.

Let $$Y$$ be dependent variable and $$X_i$$ - independent. Then $$Y={0,1}$$ and $$X_i={0,1,2}$$. But I always can split variable $$X_i$$ in this way: $$X_i:=X_i'+X_i'$$ which both is i.i.d. and take values $${0,1}$$.

– EdM
Nov 12 '16 at 13:22
• Look up the group LASSO. Nov 12 '16 at 14:25
• The use of "$Y$" for the independent variable (a regressor) and "$X_i$" for dependent variables (the responses) is so unusual that I have to wonder whether you have reversed the terms "dependent" and "independent". If you have not, you have a multivariate regression problem. If you have, then the usual Elastic Net works just fine.
– whuber
Nov 12 '16 at 15:22
• Oh, god, excuse me, you are absolutely right, it was mistake, thank you.
– Kess
Nov 12 '16 at 17:02

I am not sure if you are familiar with R, put the R package glmnet containts "extremely efficient procedures for fitting the entire lasso or elastic-netregularization path for linear regression, logistic and multinomial regression models, Poisson regression and the Cox model".

The general syntax to "Fit a generalized linear model via penalized maximum likelihood" would be:

fit=glmnet(x,y,family="binomial")

where x is your input matrix of independent variables, and y is your dependent variable (response variable). The binomial family would be for your binary dependent variable or family "multinomial" for a multinomial dependent variable.

• Thank you, it is very useful answer, because I'm using exactly R in my studies.
– Kess
Nov 13 '16 at 10:48

For categorical independent variables, ridge and lasso regression work fine with the usual OLS coding strategies, such as dummy coding. For categorical dependent variables, you can use ridge or lasso penalties with logistic regression or probit regression.

• Agreed, but a decision should be made consciously about whether the dummy-coded variables are standardized to zero mean and unit standard deviation. That is the default in some implementations, but might not always be the best choice for dummy variables. Harrell discusses this in Chapter 9 of Regression Modeling Strategies.
– EdM
Nov 13 '16 at 3:26