2
$\begingroup$

Assume we have a simple linear regression model expressed as $Y= X \beta + e$.

We know that finding the regression coefficients $\beta$ using the LASSO method is performed by penalizing the Least Squares by the L1 norm penalty:

$~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~$$argmin_\beta~(Y - X\beta)^2 + \lambda ||\beta||_1$

The most known technique to find the parameter $\lambda$ is k-fold cross validation. So can anyone help me how can I apply in matlab the k-fold cross validation in order to find the values of $\lambda$?

Any help will be very appreciated!

$\endgroup$
  • $\begingroup$ $[B ~~FitInfo] = lasso(X,Y,'CV',k);$ $%Plot the cross-validated fits.$ $lassoPlot(B,FitInfo,'PlotType','CV');$ $\endgroup$ – Christina Sep 2 '15 at 13:34
3
$\begingroup$

If you have N samples, then your Y matrix is of size N x 1 and your X matrix is of size N x D.

To optimize the regularization parameter $ \lambda $, here is a way to do k-fold cross-validation in MATLAB :

groups = crossvalind('Kfold', N, K);

groups is a 1 x N vector, and it has values between 1 and K. So for each fold, you get training and test samples with:

for k=1:K
    trains = find(groups~=k);
    tests = find(groups==k);
end

Since your case is regression, you train your system (for each fold) like this :

beta = inv(X(trains,:)) * Y

and you get your test set estimates like this :

Y_test = X(tests,:) * beta

For optimization you need a measure of accuracy, which might be RMSE for regression. So you need to try different values of lambda, by looking at the error (average of K folds), you can choose the lambda that results in the minimum average error.

Hope it helps !

$\endgroup$
1
$\begingroup$

You can simply use lasso function in MATLAB with 'CV'=k. In the output FitInfo you can find everything you want. For details, please read the MATLAB documentation.

$\endgroup$
  • 1
    $\begingroup$ We are looking for long answers. Please extend your answer. $\endgroup$ – Ferdi Jan 20 '17 at 17:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.