0
$\begingroup$

I have two different tests that measure the skill of participants in two different fields, leading to two different test scores for each participant. I want to combine both test scores to get an overall test score for each participant.

I have a guide that suggests to do the following:

  • Step 1: Compute the skill percentile compared to other participants for each participant separately for each of the two tests.
  • Step 2: Take the average of both test percentiles for each participant.

For instance, person X has a test score percentile of 70 in test A and a percentile of 60 in test B, leading to a final test score of 65.

I'm wondering if this approach is leading to any bias, since I was assuming that you shouldn't just take the average of percentiles. Is it valid to take the average of two percentiles as described above?

$\endgroup$
1
$\begingroup$

Percentiles or average ranks can actually be decent summaries when you have perhaps more than two tests. For two tests the adequacy of averaging percentiles is less clear. But first make sure that percentiling for one test is a good idea. Percentiling implies that the absolute scores are not properly quantifying an attribute, and that the appropriate way to judge one individual is only in regards to how that individual competes with others. When you want to reward or instill competition, use percentiles.

$\endgroup$
  • $\begingroup$ Thanks a lot for your response! I'm happy to hear that average percentiles might be a good idea. Do you maybe know any literature/link that discusses this method in some more detail? Unfortunately, I didn't find any good reference (maybe I'm searching for the wrong terms). $\endgroup$ – Joachim Schork Nov 24 '20 at 12:53
  • $\begingroup$ I'm not trying to imply that averaging two percentiles is a good idea. $\endgroup$ – Frank Harrell Nov 24 '20 at 13:21
  • $\begingroup$ OK thanks for clarifying, seems like I got you wrong before. In case you know any reference discussing this topic please let me know. $\endgroup$ – Joachim Schork Nov 24 '20 at 14:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.