I'm trying to solve the following problem. Multiple people score multiple vendors. The problem is that some people are less critical than others which results in them giving higher scores in general. Both #people and #vendors is low. Thus if you just sum all the scores for one vendor, the person being more generous has a bigger impact on the outcome.
The strategy I applied first was:
- calculate z score within one person's scores
- Map to 0 - 1 range within person's scores
- Divide each score of person by sum of his scores.
This way I keep the proportionality of their scores and the total "points" they gave is 1, making it equal for everyone.
Is this the correct way to tackle this problem or not?