I want to do the following:
1) OLS regression (no penalization term) to get beta coefficients $b_{j}^{*}$; $j$ stands for the variables used to regress. I do this by
lm.model = lm(y~ 0 + x)
betas = coefficients(lm.model)
2) Lasso regression with a penalization term, the selection criteria shall be the Bayesian Information Criteria (BIC), given by
$$\lambda _{j} = \frac{\log (T)}{T|b_{j}^{*}|}$$
where $j$ stands for the variable/regressor number, $T$ for the number of observations, and $b_{j}^{*}$ for the initial betas obtained in step 1). I want to have regression results for this specific $\lambda_j$ value, which is different for each regressor used. Hence if there are three variables, there will be three different values $\lambda_j$.
The OLS-Lasso optimization problem is then given by
$$\underset{b\epsilon \mathbb{R}^{n} }{min} = \left \{ \sum_{t=1}^{T}(y_{t}-b^{\top} X_{t} )^{2} + T\sum_{j=1}^{m} ( \lambda_{t}|b_{j}| )\right \}$$
How can I do this in R with either the lars or glmnet package? I cannot find a way to specify lambda and I am not 100% sure if I get the correct results if I run
lars.model <- lars(x,y,type = "lasso", intercept = FALSE)
predict.lars(lars.model, type="coefficients", mode="lambda")
I appreciate any help here.
Update:
I have used the following code now:
fits.cv = cv.glmnet(x,y,type="mse",penalty.factor = pnlty)
lmin = as.numeric(fits.cv[9]) #lambda.min
fits = glmnet(x,y, alpha=1, intercept=FALSE, penalty.factor = pnlty)
coef = coef(fits, s = lmin)
In line 1 I use cross validation with my specified penalty factor ($\lambda _{j} = \frac{\log (T)}{T|b_{j}^{*}|}$), which is different for each regressor.
Line 2 selects the "lambda.min" of fits.cv, which is the lambda that gives minimum mean cross-validation error.
Line 3 performs a lasso fit (alpha=1
) on the data. Again I used the penalty factor $\lambda$.
Line 4 extracts the coefficients from fits which belong to the "optimal" $\lambda$ chosen in line 2.
Now I have the beta coefficients for the regressors which depict the optimal solution of the minimization problem
$$\underset{b\epsilon \mathbb{R}^{n} }{min} = \left \{ \sum_{t=1}^{T}(y_{t}-b^{\top} X_{t} )^{2} + T\sum_{j=1}^{m} ( \lambda_{t}|b_{j}| )\right \}$$
with a penalty factor $\lambda _{j} = \frac{\log (T)}{T|b_{j}^{*}|}$. The optimal set of coefficients is most likely a subset of the regressors which I initially used, this is a consequence of the Lasso method which shrinks down the number of used regressors.
Is my understanding and the code correct?
$\alpha$
becomes $\alpha$. Please make this, as it will make people more easily able to understand your question, and therefore answer it. $\endgroup$