I want to calculate the AUC and also boundaries of a 90%, 95% and 99% confidence interval based on the percentiles of the block bootstrapped sampling distribution of the AUC statistic. As block bootstrap procedure, I want to use the "stationary bootstrap" developed by Politis and Romano (1994).

I have one time series, which gives probabilities between 0 and 1, and another time series which has either 0 or 1 as values.

I try to do this in R and this is what I have done so far:

# set the number of replications for the block bootstrap:
reps <- 1000
# set the block length:
block_length <- 96
# write the AUC function for the block boostrap command:
block.boot.function <- function(x){

# block bootstrap for AUC:
auc_block_boot <- tsboot(tseries = data_ROC$y , statistic = block.boot.function, R = reps, l = block_length, sim = "geom")

> auc_block_boot


Average Block Length of 96 

tsboot(tseries = data_ROC$y, statistic = block.boot.function, 
    R = reps, l = block_length, sim = "geom")

Bootstrap Statistics :
     original     bias    std. error
t1* 0.9694139 -0.3090732   0.1147533

# Confidence intervals based on the percentiles: 
auc_block_boot_CI <- boot.ci(auc_block_boot, conf = c(0.9, 0.95, 0.99), type = c("perc"))
> auc_block_boot_CI
Based on 1000 bootstrap replicates

boot.ci(boot.out = auc_block_boot, conf = c(0.9, 0.95, 0.99), 
    type = c("perc"))

Intervals : 
Level     Percentile     
90%   ( 0.4945,  0.8667 )   
95%   ( 0.4723,  0.9018 )   
99%   ( 0.4421,  0.9506 ) 
Calculations and Intervals on Original Scale
Some percentile intervals may be unstable

I am unsure whether the code reproduces what I described in the beginning. The relatively large bias of -0.3090732 makes me skeptical that the results are okay. The original AUC value 0.9694139 is also outside the confidence intervals. Is this an error that shows that the COde is wrong, or is this possible with percentiles?

I would be glad about any help!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.