Multiple testing following ANCOVA, or more generally any GLM, but the comparisons now focus on the adjusted group/treatment or marginal means (i.e. what the scores would be if groups did not differ on the covariate of interest). To my knowledge, Tukey HSD and Scheffé tests are used. Both are quite conservative and will tend to bound type I error rate. The latter is preferred in case of unequal sample size in each group. I seem to remember that some people also use Sidak correction on specific contrasts (when it is of interest of course) as it is less conservative than the Bonferroni correction.
Such tests are readily available in the R
multcomp package (see
?glht). The accompagnying vignette include example of use in the case of a simple linear model (section 2), but it can be extended to any other model form. Other examples can be found in the
HH packages (see
?MMC). Several MCP and resampling procedures (recommended for strong inferences, but it relies on a different approach to the correction for Type I error rate inflation) are also available in the
multtest package, through Bioconductor, see refs (3–4). The definitive reference to multiple comparison is the book from the same authors: Dudoit, S. and van der Laan, M.J., Multiple Testing Procedures with Applications to Genomics (Springer, 2008).
Reference 2 explained the difference between MCP in the general case (ANOVA, working with unadjusted means) vs. ANCOVA. There are also several papers that I can't remember actually, but I will look at them.
Other useful references:
- Westfall, P.H. (1997). Multiple Testing of General Contrasts Using Logical Contraints and Correlations. JASA 92: 299-306.
- Westfall, P.H. and Young, S.S. (1993) Resampling Based Multiple Testing, Examples and Methods for p-Value Adjustment. John Wiley and Sons: New York.
- Pollard, K.S., Dudoit, S., and van der Laan, M.J. (2004). Multiple Testing Procedures: R multtest Package and Applications to Genomics.
- Taylor, S.L. Lang, D.T., and Pollard, K.S. (2007). Improvements to the multiple testing package multtest. R News 7(3): 52-55.
- Bretz, F., Genz, A., and Hothorn, L.A. (2001). On the numerical availability of multiple comparison procedures. Biometrical Journal, 43(5): 645–656.
- Hothorn, T., Bretz, F., and Westfall, P. (2008). Simultaneous Inference in General Parametric Models. Department of Statistics: Technical Reports, Nr. 19.
The first two are referenced in SAS PROC related to MCP.