I am not sure whether or not this problem can be solved through statistical methods, but here is the problem:
I have 4 professional reference providers for a job candidate rating the candidate on a scale of 0-100 on say 10 items. Behind the scenes we check for the consistency of ratings for each item from the 4 reference providers and arrive at a total score for each of the 10 items.
The problem is what is the 4 reference proviers collude with the candidate and give a rating of 100/100 on every item? Such scores cannot be prevented because there may really be an excellent candidate who can score 100/100 on every item from each of the 4 reference providers.
So, is it possible to design a statistical method to differentiate between the fake candidate colluding with reference providers and the excellent candidate getting all 100 scores?
