8
$\begingroup$

When doing ANCOVAs, software usually gives both unadjusted means as well as adjusted means. However, the post hoc tests for most statistical programs are limited to unadjusted means. Yet, sometimes, if one is doing an ANCOVA, post hoc comparisons of adjusted means is what is desired, not the unadjusted comparisons. Or am I mistaken – should post hoc tests of adjusted means be avoided for some reason?

Post hoc tests for adjusted means have been around for quite some time, such as the GT2 method of Hochberg (1974). A modified version of the GT2 test appears in Sokal and Rohlf's Biometry (1981 2nd edition, and presumably in newer versions) specifically for adjusted means. There is also the T-method (Spjotvoll and Stoline 1973), also modified in Sokal and Rohlf (1981) for comparing adjusted means.

Why then do statistical programs not give post hoc tests for adjusted means, if one wants to know which treatment (A vs B, A vs C, etc) differ, after removing some of the variance of the covariate?

Hochberg, Y. 1974. Some Conservative Generalizations of the T-Method in Simultaneous Inference. Journal of Multivariate Analysis, 4:224 -234.

Spjotvoll, E, Stoline, MR. 1973. An extension of the T-method of multiple comparison to include the cases with unequal sample sizes. J Am Stat Assoc. 68:976-978.

Improve this question
$\endgroup$
7
  • $\begingroup$ Not the worst question ever (i.e., +1), but I somehow question that most post-hoc test report unadjusted means for the covariate. Can you give an example for at least one software? $\endgroup$
    – Henrik
    Sep 24 '12 at 22:29
  • $\begingroup$ I think you make a good argument. Sometimes post hoc analyses are done on subpopulations in which case the conditional information relative to the subpopulation would justify doing that analysis on adjusted means. However in clinical trials the only thing that really matter regarding say FDA approval of a drug is the treatment difference on the entire target population. Post hoc subgroup analysis may be accepted as good exploratory information. But it would not be accepted for labeling purposes. To make a claim on subgroups would require an independent prospective trial. $\endgroup$
    – Michael R. Chernick
    Sep 24 '12 at 22:30
  • $\begingroup$ @Henrik, Statistica and SPSS done all posthoc tests using the non-adjusted means. I agree that one have to be very careful when running posthoc after ANCOVA since most of the softwares that do post hoc comparison after ANCOVAs do it BUT on non-adjusted means. Bryan Paulson Tukey (BPT) test is recommended for pairwise comparison on ADJUSTED means, another procedure could be the conditional Tukey Kramer test. The SPSS has a pariwise comparison using adjusted means. $\endgroup$
    – AnastD
    Sep 25 '12 at 0:32
  • $\begingroup$ @Henrik, According to the Statistica website, "STATISTICA will always compute post-hoc tests using observed means, taking the estimate of Sigma (when appropriate, i.e., when called for by the respective test) from the overall analysis (ANOVA). For ANCOVA designs, even though all post-hoc tests are performed on observed means, the estimate of Sigma for the tests will be "adjusted" by the presence of covariates in the model (because the MS-error for the between-group design will have been effectively adjusted)." $\endgroup$
    – P auritus
    Sep 25 '12 at 12:54
  • $\begingroup$ @Michael Chernick, I was hoping for a more general consideration. Say I was interested in comparing concentrations of a certain protein in animals on different diets, but there was a range of ages among the animals, age also varied among treatments, and protein concentration was proportional to age. If I want to know the differences in protein concentrations between diets, I could have age as a covariate. If I wanted to make claims whether diet A caused lower protein than diet B and C, etc, independent of age, would I not want to use post hoc tests on adjusted means? $\endgroup$
    – P auritus
    Sep 25 '12 at 13:08
2
$\begingroup$

Did I understand that you were looking for a software capable of calculating ANCOVA based on adjusted means? There is a free statistical software called PAST that does use adjusted means for ANCOVA.

It is available at http://folk.uio.no/ohammer/past/

Improve this answer
$\endgroup$
0
$\begingroup$

In STATISTICA (StatSoft), when performing ANCOVA, post-hoc values are related of adjusted means. Even though in top-X axis are present the "non-adjusted means".

Do the test... compare means and do the post-hoc with and without the covariates... P values are different.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.