4
$\begingroup$

In the multiple testing problem for independent samples, general procedure is one way ANOVA followed by Tukey's HSD or Scheffe’s Method.

For multiple dependent samples, one way ANOVA is replaced by repeated measure ANOVA. But what is the analogy of Tukey or Scheffe in the dependent sample setting?

$\endgroup$
1
  • 1
    $\begingroup$ Unfortunately, the terminology 'repeated measures' has been used (and abused) in so many fields of application to mean so many different things that one has to supply context whenever 'repeated measures' is used. I think I know what you want. Look at my Answer. I hope it is useful. If not, please supply additional details and maybe a more useful answer will appear. $\endgroup$
    – BruceET
    Apr 21, 2020 at 2:29

1 Answer 1

0
$\begingroup$

In statistical design the generalization of pair is block, so I think you are looking for a block design.

For example, if three types of chocolate are being rated for flavor by a panel of 15 professional tasters, then you would have a two-way ANOVA with factors Type and Taster, and one Flavor score for each Type by each Taster. (Altogether: 45 observations.) One hopes that professional Tasters have more or less fixed standards so that the variation among them is not a major issue of the data analysis.

The main goal is to see if there are statistically significant differences among Types. The main F-test for Type establishes that factor as significant. The bottom line for you is that there is a way to apply Tukey's HSD method to Types for post hoc analysis of this two-way ANOVA. [Because there is only one rating score per cell, this model does not have an interaction term.]

This R-Bloggers page shows which R procedures to use and the syntax for using them. [The example, originally written in Portuguese and translated to English, retains Portuguese variable names: Sabor for flavor score, Tipo for type of chocolate, and Provador for taster.]

$\endgroup$
4
  • $\begingroup$ Thanks for that link! Yes, what I meant repeated measure is essentially two-way ANOVA. I understand this two-way ANOVA can serve as an extension of paired t-test for multiple groups. But is this Tukey's HSD justified for dependent groups ? My understanding is that this Tukey's HSD after two-way ANOVA just use a better estimate of the variance to calculate the CI but does not account for dependent groups. $\endgroup$
    – Statisfun
    Apr 21, 2020 at 17:38
  • $\begingroup$ IMHO: HSD for 2-way treats comparisons among Types differently than if Taster effect not included. In 2-way model SS(Error), hence MS(Error) smaller than in 1-way just for Types wrongly ignoring Tasters. HSD in link OK.// Controversy arises if there're several rounds of tasting giving > 1 reps per cell. Then Tasters is recognized as random effect and denom term for F-test on Types would depend on whether restricted or unrestricted model used. // Should be no ad hoc testing for signif among levels of rand effect. Instead should estimate Taster variance, which requires comp-intensive methods. $\endgroup$
    – BruceET
    Apr 21, 2020 at 18:46
  • $\begingroup$ My question is what if there is correlation between chocolate brands within each taster ? For the Tukey's HSD to work, the two brands have to be independent, right ? Is Tukey HSD still valid under this dependence ? $\endgroup$
    – Statisfun
    Apr 22, 2020 at 0:36
  • $\begingroup$ No nesting in a block design with one obs per cell. Can't make sense of brands within taster. Please tell me what you mean in terms of your design. Exactly what to you fear may be dependent? Have you written a formal model for your design? If so, maybe you can put it into your Question for all to see. As is, your question is quite sparse and only hints at what seems to be bothering you. Site discourages chatting in Comments. Please edit question for clarity. $\endgroup$
    – BruceET
    Apr 22, 2020 at 2:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.