Given the number of users of an application was 70 in total, it's my understanding that research shows:
Five users is the number of users needed to detect approximately 85% of the problems in an interface, given that the probability a user would encounter a problem is about 31%.
Thing is that over time, it seems likely that a given user's abilities to detect problems compared to other users would likely become more predictable. Is this true, and if not why? If it is true, what would be the best formula for knowing when to switch from random selection to optimal selection?
Note: Please note that this question is roughly understood by me, and it's very possible that there's a fundamental issue with the premise and/or gaps in the assumptions provided. If so, please comment and I'll attempt to address any concerns, questions, etc.