Advantages of ROC curves What is the advantages of the ROC curves?
For example I am classifying some images which is a binary classification problem.
I extracted about 500 features and applied a features selection algorithm to select a set of features then I applied SVM for classification. In this case how can I get a ROC curve?
Should I change the threshold values of my feature selection algorithm and get sensitivty and specificity of the output to draw a ROC curve?
In my case what is the purpose of creating a ROC curve?
 A: Many binary classification algorithms compute a sort of classification score (sometimes but not always this is a probability of being in the target state), and they classify based upon whether or not the score is above a certain threshold.  Viewing the ROC curve lets you see the tradeoff between sensitivity and specificity for all possible thresholds rather than just the one that was chosen by the modeling technique.  Different classification objectives might make one point on the curve more suitable for one task and another more suitable for a different task, so looking at the ROC curve is a way to assess the model independent of the choice of a threshold.
A: ROC curves are not informative in 99% of the cases I've seen over the past few years.  They seem to be thought of as obligatory by many statisticians and even more machine learning practitioners.   And make sure your problem is really a classification problem and not a risk estimation problem.  At the heart of problems with ROC curves is that they invite users to use cutpoints for continuous variables, and they use backwards probabilities, i.e., probabilities of events that are in reverse time order (sensitivity and specificity).  ROC curves cannot be used to find optimum tradeoffs except in very special cases where users of a decision rule abdicate their loss (cost; utility) function to the analyst.
A: After creating a ROC curve, the AUC (area under the curve) can be calculated. The AUC is accuracy of the test across many thresholds. AUC = 1 means the test is perfect. AUC = .5 means performs at chance for binary classification.
If there are multiple models, AUC provides a single measurement to compare across different models. There are always trade-offs with any single measure but AUC is a good place to start.
A: The AUC does not compare classes real vs. predicted with each other. It is not looking at the predicted class, but the prediction score or the probability. You can do the prediction of the class by applying a cutoff to this score, say, every sample that got a score below 0.5 is classified as negative. But the ROC comes before that happens. It is working with the scores/class-probabilities.
It takes these scores and sorts all samples according to that score. Now, whenever you find a positive sample the ROC-curve makes a step up (along the y-axis). Whenever you find a negative sample you move right (along the x-axis). If that score is different for the two classes, the positive samples come first (usually). That means you make more steps up than to the right. Further down the list the negative samples will come, so you move left. When you are through the whole list of samples you reach at the coordinate (1,1) which corresponds to 100% of the positive and 100% of the negative samples. 
If the score perfectly separates the positive from the negative samples you move all the way from (x=0, y=0) to (1,0) and then from there to (1, 1). So, the area under the curve is 1. 
If your score has the same distribution for positive and negative samples the probabilities to find a positive or negative sample in the sorted list are equal and therefore the probabilities to move up or left in the ROC-curve are equal. That is why you move along the diagonal, because you essentially move up and left, and up and left, and so on... which gives an AROC value of around 0.5.
In the case of an imbalanced dataset, the stepsize is different. So, you make smaller steps to the left (if you have more negative samples). That is why the score is more or less independent of the imbalance. 
So with the ROC curve, you can visualize how your samples are separated and the area under the curve can be a very good metric to measure the performance of a binary classification algorithm or any variable that may be used to separate classes.

The figure shows the same distributions with different sample sizes. The black area shows where ROC-curves of random mixtures of positive and negative samples would be expected. 
