Determining the validity of ROC curves

Question

ROC curves were generated using ROCR package using only actual and predicted labels(from a SVM) as input.

Script:

A1_prd <- prediction(A1$Original, A1$Predicted)
A1_prf <- performance(A1_prd, "tpr", "fpr")


A2_prd <- prediction(A2$Original, A2$Predicted)
A2_prf <- performance(A2_prd, "tpr", "fpr")

plot(A1_prf, col = 3)
plot(A2_prf, col = 4, add = T) 

Input:

>head(A1) # Similar for A2
  Original Predicted
1       -1         1
2       -1        -1
3       -1         1
4       -1         1
5       -1         1
6       -1        -1

Generally ROC curves have typical staircase/curve style appearance, but the plot generated by ROCR doesn't resemble it. I wonder if it is a valid ROC to infer. Please clarify.

Answer 1

Im reading about ROC curve and come across your question. For the argument predictions of function prediction you need a vector (or matrix) of probabilities and not the predicted classes. ROC curve shows you all possible thresholds, if you only habe the predicted classes, how can ROCR see the difference when it changes the threshold.

In addition i see you take the false order for the argument. ?prediction give: prediction(predictions, labels, label.ordering = NULL). That means: your command should be:

A1_prd <- prediction(A1$Predicted.Probabilites, A1$Original)

And don't forget the argument label.ordering if your response variable is not an ordered factor and the order of levels not "natural" in R.