Is there LASSO type model in which only some of the regressors are regulated? I am a beginner with LASSO.
Is there any way (paper/code) to perform a LASSO type model in which only some of the explanatory variables are regularized? Or imposing different regulation parameters to several explanatory variables groups?
 A: Yes, this is easily possible. If you use R, look into the penalized package.
A: I recommend using the glmnet package of Friedman and Tibshirani.  The functions glmnet and cv.glmnet accept a parameter, 'penalty.factor', that appears to be what you need.
A: You can do it by hand with the use of nlm in R. The caveat being that an "all purpose" optimizer such as nlm can converge to local minima or diverge and is particularly slow. Contrast with the specialized gradient descent code that Tibshirani and those guys made to estimate LASSO models: it is much more regular and ridiculously fast. Obviously, this introduces some sample size issues. With unconstrained OLS, you necessarily have to have $p \ll n$, LASSO lets you have $p \gg n$. This maximum likelihood, err maximum penalized likelihood, err maximum product likelihood penalized likelihood approach is constrained in the  OLS way.

*

*calculate residuals' log product density in the normal probability model

*subtract penalized parameter and L1 norm for LASSO'd parameters

*maximize partially penalized likelihood in nlm.

Example below:
set.seed(4)
n <- 100
p <- 10
x <- matrix(rnorm(n*p), n, p)
X <- model.matrix(~x)
y <- rnorm(n)


## i: index of which parameters aren't tuned
## l: tuning parameter
LASSO.OLS.LOVECHILD <- function(i, l) {
  penLogLik <- function(b) {
    -sum(dnorm(y - b%*%t(X), log=TRUE)) + l*sum(abs(b[!i]))
  }
  b <- vector('numeric', p+1)
  val <- nlm(penLogLik, b)
  val$estimate
}

## ordinary least squares 1 way: all unpenalized betas
LASSO.OLS.LOVECHILD(i=rep(TRUE, p), l=20)
lm(y ~ x)

## ordinary least squares another way (no tuning)
LASSO.OLS.LOVECHILD(i=rep(FALSE, p), l=0)
lm(y ~ x)

## unconstrained first half, heavy tuning forces last 5 to 0
## (not too heavy or else algorithm doesn't converge to anything...)
round(LASSO.OLS.LOVECHILD(i=1:10 %in% 1:5, l=20), 2)
round(lm(y ~ x[, 1:5])$coef, 2)

