Cross validation with two parameters: elastic net case I want to know the cross validation procedure to find the two parameters of elastic net presented by Zou and Hastie on the prostate dataset as example.
I can't improve the error rate lasso with k-fold when I use elastic net.
 A: If by error rate you mean the usual one (proportion classified correctly), this is a discontinuous improper scoring rule.  An improper scoring rule is optimized by a bogus model.  It will lead you to select the wrong features and to discard features that are predictive.  It is better to use the most sensitive measure for assessing a model, which will be based on deviance or penalized deviance.  In simpler cases where I'm only using a quadratic penalty, I solve for the optimum penalty using effective AIC.  An example is in my book Regression Modeling Strategies.  A simulation study of this approach is on http://biostat.mc.vanderbilt.edu/rms.
A: Rather than use a grid search, use a numeric optimisation routine, such as the Nelder-Mead simplex algorithm.  This is generally more efficient than grid search and it will be well worth the effort in the long run.  In MATLAB this is implemented by the fminsearch routine of the optimisation toolbox, but I expect there is an R implementation as well.  The cost function for the optimisation is simply the cross-validated performance estimate.
It is a good idea to re-parameterise the problem first so as to achieve a non-constrained optimisation problem (the regularisation parameters must be positive).  I do this by optimising the logarithm of the regularisation parameters, and have found it generally works well.
A: The method to use in this case is exactly the same, though e.g. the glmnet package doesn't provide it out of the box.
Instead of working over 1 discrete set of parameter values (lambda), you now crossvalidate for a grid of parameter values, (lambda and alpha), then pick the best value (lambda.min and alpha.min), and then the lambda and alpha so that lambda is the biggest possible but its predictive measure is within 1 SE of that of lambda.min and alpha.min.
If you use R, you can probably do something like:
alphasOfInterest<-seq(0,1,by=0.1) #or something similar
#step 1: do all crossvalidations for each alpha
cvs<-lapply(alphasOfInterest, function(curAlpha){
  cv.glmnet(myX, myY, alpha=curAlpha, some more parameters)
})
#step 2: collect the optimum lambda for each alpha
optimumPerAlpha<-sapply(seq_along(alphasOfInterest), function(curi){
  curcvs<-cvs[[curi]]
  curAlpha<-alphasOfInterest[curi]
  indOfMin<-match(curcvs$lambda.min, curcvs$lambda)
  c(lam=curcvs$lambda.min, alph=curAlpha, cvup=curcvs$cvup[indOfMin])
})
#step 3: find the overall optimum
posOfOptimum<-which.min(optimumPerAlpha["lam",])
overall.lambda.min<-optimumPerAlpha["lam",posOfOptimum]
overall.alpha.min<-optimumPerAlpha["alph",posOfOptimum]
overall.criterionthreshold<-optimumPerAlpha["cvup",posOfOptimum]
#step 4: now check for each alpha which lambda is the best within the threshold
corrected1se<-sapply(seq_along(alphasOfInterest), function(curi){
  curcvs<-cvs[[curi]]
  lams<-curcvs$lambda
  lams[lams<overall.lambda.min]<-NA
  lams[curcvs$cvm > overall.criterionthreshold]<-NA
  lam1se<-max(lams, na.rm=TRUE)
  c(lam=lam1se, alph=alphasOfInterest[curi])
})
#step 5: find the best (lowest) of these lambdas
overall.lambda.1se<-max(corrected1se["lam", ])
pos<-match(overall.lambda.1se, corrected1se["lam", ])
overall.alpha.&se<-corrected1se["alph", pos]

All this code is untested + needs attention if you use auc as your criterion (because then you need to look for the maximum of the criterion and some other details change), but the ideas are there.
Note: in the last step, you could, instead of going for the highest lambda, find the one that has the most parsimonious model (because higher lambda does not guarantee more parsimony over different alphas)
You may also want to collect all lambdas up front, and pass the collection of all those to every crossvalidation, so that you can ensure that each crossvalidation uses the same set of lambdas. This is easy to do but requires some extra steps. I'm not certain whether it is necessary...
