ROC curve for two-sided cut-off

Question

I am very, very confused about ROC curves. I have a Bayesian model which outputs a prevalence on a continuous scale between 0 and 1. I have a classification I would like to use that classifies that prevalence into an endemicity class based on a two sided cut-off (i.e. the prevalence cut off for being in the class is $ 0.35 <= P <= 0.6 $.

So, I have the true observed prevalence, the predicted prevalence from my model (which is taken as the mean of the posterior distribution for that observation) I also have a probability for each prediction, that it belongs in the endemicity class of interest. This probability is an output of my Bayesian model and is simply the number of times the predicted prevalence did fall between those endemicity cut-offs divided by the number of posterior draws from my model (so a probability of $1$ means that all posterior realisations fell between 0.35 and 0.6).

Here is an example of my data:

> head(mydat)
    Obs  Pred  pClass Class
1 0.441 0.503 0.56125     1
2 0.664 0.698 0.22225     0
3 0.252 0.468 0.58725     0
4 0.226 0.374 0.39325     0
5 0.014 0.107 0.02975     0
6 0.713 0.571 0.46425     0

Where:
Obs = the observed true prevalence
Pred = the predicted prevalence
Pclass = the proportion of posterior realisations from my model which fell within the endemicity class of interest.
Class - is the true class label.

I used the following R code to calculate what I think a ROC curve should be:

require(ROCR)
pred <- with( mydat , prediction( pred=Pclass ,labels=Class )
plot(perf <- performance(pred,'tpr','fpr') )

Producing this plot:

Have I calculated this correctly?
Have I used the right data?! Do I have the right data for calculating a ROC curve?
Can you produce a valid ROC curve for a classifier which has a two-sided cut-off?

I did read many tutorials on ROC curves, but my intuition fails me and I can't quite work out if I am applying it to my data in a valid way. Here is a copy/paste to put my data into R...

mydat <- structure(list(Obs = c(0.441, 0.664, 0.252, 0.226, 0.014, 0.713, 
0.543, 0.777, 0.472, 0.512, 0.436, 0.312, 0.403, 0.709, 0.472, 
0.625, 0.056, 0.335, 0.596, 0.679, 0.143, 0.68, 0.489, 0.319, 
0.706, 0.789, 0.14, 0.261, 0.592, 0.05, 0.736), PRed = c(0.503, 
0.698, 0.468, 0.374, 0.107, 0.571, 0.619, 0.693, 0.569, 0.584, 
0.463, 0.03, 0.363, 0.562, 0.471, 0.313, 0.44, 0.448, 0.506, 
0.617, 0.378, 0.643, 0.603, 0.403, 0.61, 0.743, 0.29, 0.666, 
0.685, 0.044, 0.625), pClass = c(0.56125, 0.22225, 0.58725, 0.39325, 
0.02975, 0.46425, 0.359, 0.2315, 0.4135, 0.419, 0.587, 0, 0.47025, 
0.38625, 0.48825, 0.32425, 0.4725, 0.5615, 0.43975, 0.3605, 0.40825, 
0.313, 0.411, 0.48525, 0.35475, 0.13475, 0.2725, 0.26875, 0.245, 
5e-04, 0.316), Class = c(1, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 0, 
1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0)), .Names = c("Obs", 
"Pred", "pClass", "Class"), row.names = c(NA, -31L), class = "data.frame")

Answer 1

Can you produce a valid ROC curve for a classifier which has a two-sided cut-off?

No, not directly. To compute an ROC curve you have to be able to sort examples based on the confidence you have that they belong to the positive class. Sorting based on the score your models output doesn't do that.

You could attempt to shoe-horn your predictions into such a level of confidence, for instance by using distance to the center of your positive interval as a criterion, but usually such hacks are a bad idea ...