# Tag Info

16

Anne, I will shorty explain how to do such multiple comparisons in general. Why this doesn't work in your specific case, I don't know; I'm sorry. But normally, you can do it with the multcomp package and the function glht. Here is an example: mydata <- read.csv("http://www.ats.ucla.edu/stat/data/binary.csv") mydata$rank <- factor(mydata$rank) my....

12

This is not an area where there is universal agreement. My view is that 1) the two tests answer different questions, so it's not surprising that they get different answers. 2) This discrepancy is more a demonstration of the problems of p values and, especially, using cutoff values like p < .05. 3) It also gets at the problems of looking for "cookie ...

12

I would say response ~ brightness+duration+(duration|subject) would probably be a little better. (The simpler (1|duration:subject) model is not necessarily wrong, but might be oversimplified. If I were a peer reviewer of this work I would certainly ask for a justification of the simpler model ...) The (duration|subject) model is a "random-slopes" model, ...

12

As far as the notched boxplot goes, the McGill et al  reference mentioned in your question contains pretty complete details (not everything I say here is explicitly mentioned there, but nevertheless it's sufficiently detailed to figure it out). The interval is a robustified but Gaussian-based one The paper quotes the following interval for notches (...

11

No, it is not a valid nonparametric alternative. The rank sum test (either original Wilcoxon flavor, or New Improved Mann-Whitney $U$ varieties): ignore the rankings used by the Kruskal-Wallis test, and do not employ pooled variance for the pairwise tests. See, for example, Kruskal-Wallis Test and Mann-Whitney U Test. (Also the pairwise.wilcox.test seems ...

10

Answer to question 1 You need to adjust for multiple comparisons if you care about the probability at which you will make a Type I error. A simple combination of metaphor/thought experiment may help: Imagine that you want to win the lottery. This lottery, strangely enough, gives you a 0.05 chance of winning (i.e. 1 in 20). M is the cost of the ticket in ...

10

Here's part of the table you linked to: The first few rows are obtained by: > qtukey(p = 0.95, nmeans = 2:10, df = 5)  3.635351 4.601725 5.218325 5.673125 6.032903 6.329901 6.582301 6.801398  6.994698 > qtukey(p = 0.99, nmeans = 2:10, df = 5)  5.702311 6.975727 7.804156 8.421495 8.913107 9.320875 9.668681  9.971483 10.239281 >...

9

Yes, you should use the same, transformed data throughout the analysis. Tukey's test makes the same assumptions as the ANOVA.

9

p adj is the p-value adjusted for multiple comparisons using the R function TukeyHSD(). For more information on why and how the p-value should be adjusted in those cases, see here and here. Yes you can interpret this like any other p-value, meaning that none of your comparisons are statistically significant. You can also check ?TukeyHSD and then under ...

8

It's simply that if (under the null hypothesis of no effects) there's a 5% chance, say, of a false positive in each one of 20 tests, say; there's a greater than 5% chance of a false positive in any one of those 20 tests. If the tests are independent, the change of getting a false positive in none is $95\%^{20}=35.8\%$, and so the chance of a false positive ...

8

As always, your question implicitly asks for some authoritative answer that might very well not exist. Scheffé's method and Tukey's HSD are usually called post-hoc tests, used for unplanned comparisons and conducted after an omnibus test but that's not a requirement for all such methods. The main argument for a distinction between planned and unplanned ...

6

It is certainly possible and does happen quite frequently, especially if there are many pairwise comparisons (which is likely the case if you're investigating an interaction term). The Tukey procedure controls the Type I error rate and requires a larger difference to declare significance compared to if no adjustment was used. The ANOVA F-test uses MSE in ...

6

Your original thought was incorrect. Howell's is correct. Take the simple t-test, some people just use them for planned or post hoc but adjust their p-values for multiple comparisons. Both of the tests you mention are typically used post-hoc but could be used for planned tests if the planned tests are expected to have multiple comparison issues. For ...

5

Here's a sample of 10000 normal sample range-variance pairs for $n$=30. Plainly, those aren't independent.

4

You cannot test all 3 pairwise comparisons within a single model because it will always be the case that one of the codes is a perfect linear combination of the other 2 codes. For example, in the codes you wrote, if we call the rows/contrasts $C_1$, $C_2$ and $C_3$ (from top row to bottom row), notice that $C_1 = C_2 - C_3$. On a more intuitive level, we ...

4

Your ANOVA was significant, implying you either made a Type I error or the means are not all equal (in which case the null is false). Since the chance of making a Type I error was (presumably) set fairly low, the second option becomes a relatively plausible explanation for the size of the test statistic. In that sense, the research hypothesis you stated is ...

4

If the assumption of normality for one-way ANOVA does not hold, you can turn to a nonparametric analog to the one-way ANOVA: the Kruskal-Wallis test. Just as the assumption of normality underlying the unpaired t test may not be met, thus motivating the use of the rank sum test, onne can then use Dunn's test, or the more powerful (but less well known) Conover-...

4

Try the emmeans package. Something like library(emmeans) emm = emmeans(Depth1, ~ Burn_Con * Aspect) pairs(emm) # or for simple comparisons pairs(emm, simple = “each”)

4

In a balanced design, all three of these model types will yield the same results. With unbalanced data, you will get different results for the main effects but not the interaction term, assuming you only have a two-way interaction (as in your example). The general advice is to use a Type II sums-of-squares model for unbalanced designs. This model tests the ...

4

The data are arranged in complete block design. Because the values between any two groups are paired, Tukey's HSD test isn't appropriate. Instead you want to use something like a paired t test or other tests that take into account the paired nature of the data. Using a test like Tukey's HSD or Student's t test, the fact that the data can be arranged so ...

3

First, different approaches clearly exist (for example, for post-hoc comparisons there is also Dunett's test, Scheffe's test, LSD and many others; there are non-parametric approaches; more complex modelling; bootstrapping and randomization techniques; maybe you can even drop the frequentist approach altogether and move to Bayesian factors). Whether they ...

3

I think your conclusion should be that there is a significant interaction but that this is not significant after accounting for multiple testing. But your emphasis should be on the size of the interaction and what it means rather than whether p = 0.04 or 0.07 or whatever. I would calculate the predicted level of the outcome for men and women of different ...

3

The lsmeans package makes this pretty simple: library(lsmeans) lsmeans(mod, pairwise ~ CO2 | YEAR) The tukey adjustment is the default.

3

Here is a way to generate QTable into a data frame. You can change the grid limits according to your needs. QTable <- expand.grid(alpha=c(0.01,0.05), groups=seq(2,10,1), df=seq(2,120,1)) QTable$QVal=qtukey(1-QTable$alpha,QTable$groups,df=QTable$df) head(QTable) alpha groups df QVal 1 0.01 2 ...

3

Yes, it is possible that Tukey's HSD, or any other test procedure for that matter (other than the two-sample T test), will fail to detect significant differences despite the two-sample T test / 95% confidence intervals not overlapping. The converse is also true. Fundamentally, this is because the formulae involved are different, so you can come up with ...

3

They aren't really the same. A planned comparison is something you are committing to before you see your data, and will run no matter what the results look like. A post-hoc comparison is more opportunistic. You look at that because, when you looked at the data, that particular comparison looked interesting. The idea here is that there will always be ...

3

So: Fisher LSD, Tukey HSD and Scheffe test are all parametric, so this is not an option for you p-value corrections like Bonferroni, Dunn-Sidak and many others (Holm, Benjamini-Hochberg, ...) always can be used beware of Bonferroni, it is the most conservative correction (it can dump your type-I error far below 5%) If you are looking for post-hoc test ...

3

In some situations, researchers know in advance (i.e., when they design their study) what groups they would like to compare with respect to the mean value of an outcome variable. In that case, they would not perform an omnibus ANOVA F-test but rather focus directly on performing the desired a priori group comparisons. In other situations, researchers will ...

2

There is kruskalmc function in pgirmess package in R. Description of the test: Multiple comparison test between treatments or treatments versus control after Kruskal-Wallis test.

2

JMP does Steel-Dwass comparisons. Use 'Fit Y by X' then on the 'Oneway Analysis of ...' menu choose 'Nonparametric' -> 'Nonparametric Multiple Comparisons' -> 'Steel-Dwass All Pairs'

Only top voted, non community-wiki answers of a minimum length are eligible