0
$\begingroup$

We have an annual performance metric for a number of educational institutions. The figure for any one institution varies a lot from year to year, as it depends very greatly on the student cohort, but there is a clear correlation between the metric for any two years (Pearson correlation coefficient varies from .22 to .36). Is there a statistic that we can calculate which combines several years to estimate of how much of the variation between institutions is caused by the institution itself (the remainder being due to the cohort).

[note 1: We are fully aware that not all the factors are under the control of the institution, some are dependent on the environment where the institution is situated].

[note 2: There are over 100 institutions, but only 3 years data at present (5 years soon)].

[note 3: The naive approach is to average the three years, but I hope there is something better we can do].

[note 4: I have looked at multiple correlation, but that seems to be answering a different problem (multiple factors influencing the outcome); we have multiple values for the outcome].

Thank you very much!

James

$\endgroup$

1 Answer 1

1
$\begingroup$

If you're interested in the decomposition of variance into a between-institution and within-institution components a "mixed model" (also called multilevel model) is probably what you're looking for. If your performance metric is a continuous variable and is approximately normally distributed you could try a "linear mixed model", if the metric is discrete you'll need a "generalized linear mixed model". This is a general and powerful class of methods, but it is a bit more difficult than calculating a simple summary statistic. If you can assume that each institution's performance is constant over time then a "random intercept model" would probably be the first thing to try. (And with only 3 data points it would be difficult to determine if performance is changing over time. Assuming it's constant is probably reasonable)

Without knowing a fair bit more about your data and objectives it's difficult to give much more advice, but hopefully that gets you started on your search.

$\endgroup$
1
  • $\begingroup$ Yes! Thank you, David, that sounds like pretty much perfect. Just the advice I was looking for. Things will probably go quiet at my end for a while, but once I've looked into the methods you suggest I will hopefully get back and accept this answer. Thanks again. James. $\endgroup$ Commented Jun 13, 2021 at 8:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.