1
$\begingroup$

I'm currently undertaking research creating cut-off scores using ROC curves. I have encountered some confusion regarding the outcome variable. My outcome variable can range in score from 10-50, and we are using a cut-off previously established of 20 to say whether the outcome is present or not in the individual.

Is it possible to conduct the analysis using a continuous outcome variable? Or would I need to compute another variable transforming my outcome variable into categories of 1) less than 20 and 2) greater than or equal to 20.

Additionally, I am having difficulty knowing if there are any assumptions relating to ROC curves.

Thank you for any possible help which is received.

$\endgroup$
  • $\begingroup$ I don't see your point, the ROC curve is not based on one single cut off ? The curve is constructed by plotting the hit rate against the false alarm rate for every possible value of the cut off ? $\endgroup$ – user83346 Aug 1 '17 at 6:32
  • $\begingroup$ See e.g. people.inf.elte.hu/kiss/12dwhdm/roc.pdf $\endgroup$ – user83346 Aug 1 '17 at 13:03
2
$\begingroup$

What you want is called Gini coefficient.

While it was created to measure economic inequality, it was used as the evaluation metric of a machine learning competition in a Kaggle featuring a regression task.

It shares properties with the AUC. For example, it's only concerned with the ordering of the outcomes, not their actual value. High Gini means high homogeneity between the ranks of the predictions and the actual values.

In the case of a dichotomous outcome, it's equivalent to $\text{Gini}=2\cdot \text{AUC}-1$.

$\endgroup$
1
$\begingroup$

No. ROC curves assess the performance of binary classifier systems, and by definition necessitate a binary outcome.

You can either dichotomize your continuous variable to make it binary if it makes sense (for instance if an action is taken when outcome > X, then you could use the same cut), or use an alternative way to evaluate it, like a correlation coefficient.

$\endgroup$
  • 1
    $\begingroup$ Just wanted to add to Calimo's answer. ROC curves are used for binary classification problems 99.9% of the time. However, they can be used for more than two classes but it's tedious. I've seen some work/research involving Three-Way ROCs but...they still have a ways to go before I'll bother with 'em. $\endgroup$ – Sarah W Aug 1 '17 at 12:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.