One of the most famous measurements for an information retrieval system is to compute its precision and recall. For both cases, we need to compute the number of total relevant documents and compare it with the documents that the system has returned. My question is that how can we find this super set of relevant documents in following scenario:
Consider we have an academic search engine which its job is to accept a full name of an academic paper and base on some algorithms, it returns a list of relevant papers. Here, to judge whether or not the system has a good accuracy, we wish to calculate its precision and recall. But We do not know how can we produce a set of relevant papers -which the search engine should return them, regarding to different user's query- and accordingly, computing the precision and recall.