I am trying to do some feature selection, having around 3500 variables for about 200 samples. To each sample is associated two numerical values (the expected outcome). I can't manage to make the caret work with this, or even find any information on this. Does anybody know how to do this?

As an example, my data is roughly in the following format:


S1    2.1 1.2 3.1 ... 4.2 1.7 5.2
S2    3.4 1.1 4.5 ... 5.3 1.2 5.7
S3499 2.4 3.5 5.1 ... 2.2 1.5 5.7
S3500 4.1 1.2 5.4 ... 1.2 2.1 5.8


S1    1.82 1.44
S2    2.44 1.22
S3499 1.23 1.32
S3500 1.99 1.51

Thanks, Swatchpuppy


1 Answer 1


I don't think caret supports multi-task learning in any of its functions. You could try the glmnet package, with distribution set to mgaussian. This will allow you to do feature selection via lasso regularization, ridge regularization, or elastic net regularization for a linear regression model.

There may be other R machine learning libraries with built-in feature selection that support multi-task learning. Here's some sample code for multi-task learning, using the lasso for variable selection, adapted from ?glmnet:

#Create a dataset
cf <- sample(0:1, 20, replace=TRUE) #Select some columns
response1 <- x %*% cf*runif(20) #Apply random coefficients
response2 <- x %*% cf*runif(20)
y=cbind(response1, response2)

#Fit a single lasso model
#0 for ridge
#1 for lasso
#>0 & <1 for the elastic net (mix of ridge and lasoo)

#Select lambda through cross validation
fit1m.cv <- cv.glmnet(x,y,family="mgaussian",alpha=1) 
coef(fit1m.cv) #Show coefficients at the selected value of lambda

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.