I have a dataset with a clear question I want to answer, but I'm unsure of the best statistical approach to answer my question. Here's an example that matches the structure of my data:
Let's say we have a group of ~200 physicians for whom we measure 10 comparable quality measures (e.g. adherence to guidelines), unit of measurement is a percent adherence to the measure. The question I want to ask is whether or not physicians tend to have consistently high or low adherence to all measures, or whether or not physicians have uneven adherence and tend to perform well on certain measures but not others. Ideally, the test or method I use can give me meaningful information for each individual.
I think that the 10 separate quality measures can be conceptualized as a "repeated measure" because they are clinically very similar and I except some within-subject correlation. I pondered repeated-measures ANOVA to test the null hypothesis that there is no significant difference across the quality measures; but I worry that it will be too sensitive and almost surely reject the null hypothesis and I won't learn very much, especially at the individual physician level.
I'd like to assign each physician to a percentile for the measures and perform some non-parametric test, but I'm getting out of my depth and was wondering if the CrossValidated community can help point me in the right direction. Suggestions for R packages also appreciated.