Statistical test for injury data I have patient mental and physical health scores and looking for differences pre and post injury. The injuries are categorical by anatomic site and severity. I was told to run a GEE with Walds test and everything came back significant, which doesn't make sense to me. Why should an ankle injury be as significant as a spinal injury? Since this data is collected every few years, I doubt a twisted ankle would leave significant decreased mental health scores after a few years. Something is not right.
Doing some research, I'm finding a ANCOVA or repeated measures ANOVA to be appropriate. However, when I run those tests I'm not seeing which classes are significant, just overall significance of the injury site / severity.
Can someone guide me to which is the appropriate test and if I'm using SAS, how to see significance by class?
Information about the data:
injury group = 21,400
no injury group = 5900
dependent variables are continuous Quality of Life score(can be pre and post or can calculate change in pre/post, either way continuous)
predictor variables are categorical (can be 2 variables, anatomic site + severity, or combined as 1 variable), also have some covariates
anatomic site is 8 levels (1-8)
severity is 5 levels (A-E)
so they can be combined as 1A, 1B, ... 8E
Most important information is which anatomic site / severity class level are significant and which are not.
I ran Cohens d and got little to moderate effect size. I got higher effect sizes by stratifying by anatomic site + severity (1A, 1B, ... 8E) but still highest is 0.4.
 A: First, your results show the distinction between statistical and practical significance. As comments indicate, with a large study like this it's quite possible for almost everything to be "statistically significant." The ratio of a difference to its standard error is used to evaluate statistical significance. A standard error tends to decrease with the square root of the number of observations, so even a small absolute difference can be "statistically significant" in a large study.
The question is whether a differences is large enough to matter in practice. Cohen's d tries to get to that issue by evaluating the ratio of the difference to its standard deviation. That doesn't necessarily indicate how much a difference in outcome will mean to any individual, however. I'd suggest that you pay attention to the actual estimated difference values and use your knowledge of the subject matter and the literature to evaluate how important an individual would think that differences of those absolute sizes in her own outcomes are.
What you seem to want for comparing various classes are the estimated marginal means, sometimes called LS-means. Those are model-based estimates that use all the information from the data and model, and thus are superior to stratification into subsets. Such analysis is available in SAS.
With respect to the best way to evaluate repeated measures with continuous outcomes, there are several approaches beyond GEE and repeated-measures ANOVA. Chapter 7 of Frank Harrell's course notes has an interesting summary table of those 2 methods plus 5 more, illustrating their strengths and weaknesses. The expected "effect size" is not a major reason for choosing among them.
