Using bootstrapping to calculate the AUC of a ridge regression model Edit:
I have a RNA-seq dataset with 8k genes and 100 samples (60 samples with disease condition 1 and 40 samples with disease condition 2) and I am trying to predict a dichotomous disease status.
I'd like to use ridge regression to create a model which uses the influence of all the genes within my dataset. Also, I'd like to use bootstrapping as an internal validation method to calculate the accuracy of the model with AUC. This is because I have a low amount of samples and a random hold-out method cannot be used to calculate the AUC.
This is the code that I want to use to create a ridge regression model and perform bootstrapping but I am unsure if I should use the entire dataset, a bootstrapped sample or a random split of the entire dataset.
#create ridge regression model on entire dataset or bootstrap sample
x<-model.matrix(disease_status~., df)[,-1]  
y<-ifelse(df$disease_status=="PR",1,0)
set.seed(123)
cv.ridge <- cv.glmnet(x, y, alpha = 0, family = "binomial",type.measure = "deviance") 
model <- glmnet(x, y, alpha = 0, lambda = cv.ridge$lambda.min, family = "binomial")

After this, I think I should use the predict function to calculate the accuracy of the model with AUC on bootstrapped samples but I don't know if this is correct and how to do this.
Can anyone help by guiding me in the right direction on how to correctly calculate the AUC with bootstraps of my ridge regression model?
 A: What you propose won't quite do what you want to accomplish.
When you use bootstrapping to validate a model-building process, you need to repeat all the steps of the process within each bootstrap sample. That includes identifying the penalty factor. Frank Harrell, the author of the rms package, explains that in Chapter 5 of his book and his course notes. For example:

the modeling strategy must be completely specified and then bootstrapped to get consistent estimates of variances and other sampling properties.

As I understand your code, you would be finding the penalty factor only for the full sample, then use the same penalty factor for each of the bootstrap samples with the validate() and calibrate() functions. That doesn't validate the entire process.
I don't think that the rms package provides built-in functions for including penalty-factor identification within bootstrap validation, but it's not too hard to write some simple code that can do that.
As what you are doing seems to be equivalent to a ridge logistic regression, you might find some efficiency by using the glmnet package with its cv.glmnet() function to identify the optimal ridge penalty factor for the full data set and for each bootstrap sample, as illustrated in Section 6.5.2 of An Introduction to Statistical Learning. Use a "binomial" family argument for logistic regression, and a type.measure of "mse" or "deviance" to have a strictly proper scoring rule for penalty selection.
That can be automated to return a model for each bootstrap sample at the sample's own optimal penalty factor and to test the model prediction against the full original data set. Then apply the steps in Section 5.3.5 of the Harrell references to validate and calibrate the entire model-building process.
You also might gain some efficiency by removing genes with low overall variance in expression among all samples (blinded to disease status) before proceeding with the ridge regression, and by adapting your code to the R boot() package to take advantage of parallelization.
