1
$\begingroup$

Let's say I have a dataset (data), which contains the binary target variable class and the predictions (probabilities in [0,1]).

 data<-data.frame(class=c(1,1,1,0,0,0,1,0,1,1,0,1),predict=c(0.8,0.8,0.8,0.1,0.1,0.95,0.1,0.2,0.5,0.9,0.1,0.99))
    metric <- function (D, d) {return (Hmisc::somers2(D$predict[d], D$class[d])[2])}
    #metric(data)
    boot.data <- boot::boot(data,metric, R=1000, sim="ordinary", stype="i")
    ci <- boot::boot.ci(boot.data, conf=0.95, type="basic")
    ci

The results are:

Intervals : 
Level      Basic         
95%   ( 0.0286,  1.2878 )  
Calculations and Intervals on Original Scale

I would like to ask why I get values above 1 (1.2878) for the error bar of the Gini, although Gini maximum value should be 1. I assume that there may be some kind of theoritical distribution fitted, through which the confidence interval is produced.

$\endgroup$
1
$\begingroup$

The problem is the type="basic" option used. I have no link to the original reference (Davison, A.C. and Hinkley, D.V. (1997) Bootstrap Methods and Their Application, Chapter 5. Cambridge University Press), but [as described here] the basic method relies on standard errors and therefore will not respect the boundaries of your distribution. The 'BCa' (or last resort 'percent') method should give you the desired results.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.