I've read many papers which suggest generating pairs if someone clicked on document and not clicked on some previos documents. It reasonable since user scanned not-clicked documents. But what should I do if my relevance function is already relevant in such case I will consider only mistakes of my relevance function and ignore all good examples. I don't think training on only error corpra as relevance universe can lead to good results. So I'm wondering how can I generate training pairs from correct sorting by baseline relevance function(currently on website)?