Return to Answer

f <- ols(frm, data = myd_train, method="qr")


p <- pentrace(f, seq(.2,1,by=.05))

Error in array(x, c(length(x), 1L), if (!is.null(names(x))) list(names(x),  : 
'data' must be of a vector type, was 'NULL'

 plot(p)

f <- ols(frm, data = myd_train, method="qr", x=TRUE, y=TRUE)


p <- pentrace(f, seq(.2,1,by=.05))

Error in array(x, c(length(x), 1L), if (!is.null(names(x))) list(names(x),  : 
'data' must be of a vector type, was 'NULL'

 plot(p)

Your data:

#random population of 200 subjects with 1000 variables 
    M <- matrix(rep(0,200*100),200,1000)
    for (i in 1:200) {
    set.seed(i)
      M[i,] <- ifelse(runif(1000)<0.5,-1,1)
    }
    rownames(M) <- 1:200

    #random yvars 
    set.seed(1234)
    u <- rnorm(1000)
    g <- as.vector(crossprod(t(M),u))
    h2 <- 0.5 
    set.seed(234)
    y <- g + rnorm(200,mean=0,sd=sqrt((1-h2)/h2*var(g)))

    myd <- data.frame(y=y, M)

Install rms package and load it.

This is extension of the suggestion by Frank in the comments. Dr. Harrel please correct if I am wrong (appreciate corrections).

require(rms)

ols function is used for Linear Model Estimation Using Ordinary Least Squares where can specify penalty term.

As suggested below in comments I added petrace function. This function trace AIC and BIC vs Penalty.

f <- ols(y ~ ., data = myd_train, method="qr")
p <- pentrace(f, seq(.2,1,by=.05))
plot(p)

"For an ordinary unpenalized fit from lrm or ols and for a vector or list of penalties, fits a series of logistic or linear models using penalized maximum likelihood estimation, and saves the effective degrees of freedom, Akaike Information Criterion (AIC), Schwarz Bayesian Information Criterion (BIC), and Hurvich and Tsai's corrected AIC (AIC_c). Optionally pentrace can use the nlminb function to solve for the optimum penalty factor or combination of factors penalizing different kinds of terms in the model." from rms package manual.

calibrate function is for Resampling Model Calibration and Uses bootstrapping or cross-validation to get bias-corrected (overfitting- corrected) estimates of predicted vs. observed values based on subsetting predictions into intervals. The validate function does resampling validation of a regression model, with or without backward step-down variable deletion. B = number of repetitions. For method="crossvalidation", is the number of groups of omitted observations

cal <- calibrate(f, method = "cross validation", B=20)  
plot(cal)

You can use Predict function to compute predicted values and confidence limits. I am not sure I this works in test situation.

This is extension of the suggestion by Frank in the comments. Dr. Harrel please correct if I am wrong (appreciate corrections).

require(rms)

ols function is used for Linear Model Estimation Using Ordinary Least Squares where can specify penalty term.

As suggested below in comments I added petrace function. This function trace AIC and BIC vs Penalty.

# using holdout (50% of the data) cross validation 
training.id <- sample(1:nrow(myd), round(nrow(myd)/2,0), replace = FALSE)
test.id <- setdiff(1:nrow(myd), training.id)

 myd_train <- myd[training.id,]
 myd_test  <- myd[test.id,] 

frm <- as.formula(paste("y~",paste(names(myd_train)[2:100],collapse="+")))

Important note I could not use all 1000 of the variables as the program complains if number of variable exceeds 100. Also y~. type formula designation did not work. So see above way of doing same creating formula object frm

f <- ols(frm, data = myd_train, method="qr") 


p <- pentrace(f, seq(.2,1,by=.05))

Error in array(x, c(length(x), 1L), if (!is.null(names(x))) list(names(x),  : 
'data' must be of a vector type, was 'NULL'

 plot(p)

"For an ordinary unpenalized fit from lrm or ols and for a vector or list of penalties, fits a series of logistic or linear models using penalized maximum likelihood estimation, and saves the effective degrees of freedom, Akaike Information Criterion (AIC), Schwarz Bayesian Information Criterion (BIC), and Hurvich and Tsai's corrected AIC (AIC_c). Optionally pentrace can use the nlminb function to solve for the optimum penalty factor or combination of factors penalizing different kinds of terms in the model." from rms package manual.

calibrate function is for Resampling Model Calibration and Uses bootstrapping or cross-validation to get bias-corrected (overfitting- corrected) estimates of predicted vs. observed values based on subsetting predictions into intervals. The validate function does resampling validation of a regression model, with or without backward step-down variable deletion. B = number of repetitions. For method="crossvalidation", is the number of groups of omitted observations

cal <- calibrate(f, method = "cross validation", B=20)  
plot(cal)

You can use Predict function to compute predicted values and confidence limits. I am not sure I this works in test situation.