I administer technical interviews for a programming position. During the process I generate a single number 0-12 for each applicant's likelihood to succeed within the company. Over the last year I have an ordered list of 50 or so such numbers.
I suspect that over time, my judgment has changed and I am grading people easier (or harder) now than I was a year ago. What statistical analysis would I preform to get more information on this hypothesis? Since this is largely for my own curiosity, I would ideally like a tool that generates further insight rather than a simple hypothesis rejected/not rejected.
I realize that I could do something like take the first and last thirds and do a t-value test to check the likelihood both come from the same population, but this seems intuitively like it would be the wrong approach as it would be discarding the information that contiguous numbers are very likely from the same population.
PS. Yes, I understand the limitations of generating a number for such a thing at all, and the general fallibility of "expert judgement". The entire reason for gathering these numbers to begin with is so that I can start using something other than gut feel for improving our process, but I am very conscious of the tendency to trust data like this too much.
Full disclosure, I originally posted this on the math stackexchange but I just found this one and it seems the better location for it.