# How to form a Precision-Recall curve when I only have one value for P-R?

I have a data mining assignment where I make a content-based image retrieval system. I have 20 images of 5 animals. So in total 100 images.

My system returns the 10 most relevant images to an input image. Now I need to evaluate the performance of my system with a Precision-Recall curve. However, I do not understand the concept of a Precision-Recall curve. Let's say my system returns 10 images for a gorilla image, but only 4 of them are gorillas. The other 6 images returned are other animals'. Thus,

• precision is 4/10 = 0.4 (relevants returned) / (all returned)
• recall is 4/20 = 0.2 (relevants returned) / (all relevants)

So I only have a point, <0.2,0.4>, not a curve. How do I have a curve (i.e., a set of points)? Should I change the number of images returned (this is fixed at 10 in my case)?

• Most models assign a probability of belonging to a class, not a class itself - or you squeeze one out of a classifier. The curve is derived by changing the probability cut-off. You'll likely get more detailed answers if you mention the classifier your using. Apr 17 '14 at 21:52
• I compute feature vectors (color, texture and shape) and obtain similarity scores for each, sum them up for a total similarity score, then sort descending. the top 10 image indices are the most relevant ones. I can obtain the class index from the image index since the images are ordered (20 gorillas, 20 giraffes etc.) I hope I made myself clear, since I don't fully understand the concepts classifier / descriptor etc.
– jeff
Apr 17 '14 at 21:56
• Realized I didn't read question well. Thought you had a two class problem (gorilla/no-gorilla). With more classes way beyond me, this may be helpful: stats.stackexchange.com/questions/2151/… Apr 18 '14 at 2:03

If you dispose of decision values you define a set of thresholds on said decision values. These thresholds are different settings of a classifier: e.g. you can control the level of conservatism. For logistic regression, the default threshold would be $f(\mathbf{x}) = 0.5$ but you can go over the entire range of $(0, 1)$. Typically, the thresholds are chosen to be the unique decision values your model yielded for the test set.
At each choice of threshold, your model yields different predictions (e.g. different number of positive and negative predictions). As such, you get a set of tuples with different precision and recall at every threshold, e.g. a set of tuples $( T_i, P_i, R_i )$. The PR curve is drawn based on the $( P_i, R_i )$ pairs.