1
$\begingroup$

I have a ROC curve like the figure below: enter image description here

What can be inferred given this kind of curve in general?

What are the differences of states 1, 2 and 3 in the figure?

What causes the diagram to jump straight from state 1 to state 3? Most of the ROC curves on the internet look like the figure below:

enter image description here

Improve this question
$\endgroup$
11
  • $\begingroup$ I gave a general answer. If you could add details about your data (number of samples above all) I could probably detail it some more. $\endgroup$
    – Firebug
    Commented Aug 17, 2016 at 14:26
  • 2
    $\begingroup$ Probably your model predictions only have 2 values. $\endgroup$
    – Sycorax ♦
    Commented Aug 17, 2016 at 14:31
  • $\begingroup$ @GeneralAbrial Yes.It is actually a binary classifier. or what do you mean by 2 values? $\endgroup$
    – user126608
    Commented Aug 17, 2016 at 14:35
  • 3
    $\begingroup$ I mean that when you present some data to the classifier to make predictions, it gives either A or B as an output, rather than a continuous outcome in some range. For example, logistic regression gives output in (0,1), so any real number between 0 and 1 (such as 0.2, 0.123123, or 0.996) is a legitimate prediction. But your model might only give 0.0 or 1.0. $\endgroup$
    – Sycorax ♦
    Commented Aug 17, 2016 at 14:37
  • 1
    $\begingroup$ Could you post a small samples of the predicted probabilities? Or perhaps a histogram. $\endgroup$
    – Firebug
    Commented Aug 17, 2016 at 14:43

3 Answers 3

Reset to default
2
$\begingroup$

I gather from the comments that you only have 2 unique predicted values. When you present some data to the classifier to make predictions, it gives either A or B as an output, rather than a continuous outcome in some range. For example, logistic regression gives output in (0,1), so any real number between 0 and 1 (such as 0.2, 0.123123, or 0.996) is a legitimate prediction. But your model might only give 0.0 or 1.0.

Because your ROC curve must go from (0,0) to (1,1) and be non-decreasing, and you have 2 unique prediction values, you can only have 2 points in between. Thus, the plot you have.

Most ROC curves are produced by models that give continuous outcomes. Because there are more decision points, there are more opportunities to measure error rates rates. That produces the kind of ROC curve in the second figure. By way of contrast, for example, logistic regression gives output in (0,1), so any real number between 0 and 1 (such as 0.2, 0.123123, or 0.996) is a legitimate prediction. But your model might only give 0.0 or 1.0.

Improve this answer
$\endgroup$
3
  • $\begingroup$ @ General Abrial Is my case considered to be a problem?if so how can I fix it? $\endgroup$
    – user126608
    Commented Aug 17, 2016 at 14:57
  • 1
    $\begingroup$ If you're using an SVM, you need the signed distances from the hyper plane. Read the documentation. $\endgroup$
    – Sycorax ♦
    Commented Aug 17, 2016 at 15:01
  • $\begingroup$ If the output was only {0,1} the ROC would have three points (there are three possible thresholds with only two output values), so that point is invalid. $\endgroup$
    – Firebug
    Commented Aug 17, 2016 at 15:17
2
$\begingroup$

You have four point estimates of sensitivity and specificity. This could be due to really grouped estimates or because you have too few samples.

Improve this answer
$\endgroup$
1
$\begingroup$

When calculating the AUC, you need to provide a range of values (as @General Abrial mentioned), not the class labels (0/1).. so you need to find how to get the non thresholded output from your SVM. For instance in scikit learn you would use decision_function, rather than the predict function

http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.