0
$\begingroup$

I have data from a study with three conditions. The independent variable was within-subjects and within-items.

I did an F1 and F2 analysis (= twice a repeated measures ANOVA) on the averages, by subject (the participants) and by item (the words). I checked/corrected for sphericity and the means do differ significantly.

Now I want to know how the three conditions differ from one another. I have a "Pairwise Comparisons" box and p-values (I checked "Bonferroni" when doing the analyses). But I do not have t-values.

How can I know my corresponding t-values? Or should I do some paired-samples t-tests? If so, what does this "Pairwise Comparisons" box tell me?

Screenshot of the "PC" box:
enter image description here

$\endgroup$
  • $\begingroup$ Could you clarify your 2nd sentence, and also what you mean by "item." Also, for people to interpret your software program's output, it would be best if you could show it. $\endgroup$ – rolando2 May 3 '12 at 13:01
  • $\begingroup$ I didn't want to go in too deep, because I think it's irrelevant. But is it better now? I added a screenshot as well. $\endgroup$ – Mien May 3 '12 at 13:07
  • 1
    $\begingroup$ I am not familiar with the software but if Std. Error ist the standard error of the estimate of the differences in means as I expect, the t-value is just the difference divided through the std. error (about 3.08 in the first row). $\endgroup$ – Erik May 3 '12 at 13:31
  • $\begingroup$ Someone check me on this, as I am not certain but I believe that within GLM options, the pairwise comparisons are reporting paired sample t-tests (with adjustments as requested). To get the actual df and test statistic you have to run a paired sample t-test. I guess the comparisons are helpful in that it will adjust your p and your confidence intervals? $\endgroup$ – user49191 Jun 29 '14 at 3:42
  • 1
    $\begingroup$ You must divide the Mean difference(I-J) by the Standard Error. The result is your t value. $\endgroup$ – user54690 Aug 25 '14 at 22:33
1
$\begingroup$

Bonferroni is a bound on the familywise error rate (FWER). It just involves the p-values of the individual tests. There is no test statistic for it. Basically if you are doing $k$ tests and all the $p$-values are less than $p$ Bonferroni gives $k\cdot p$ as the upper bound on FWER. It has the advantage of being general (not requiring any specific assumptions). But the disadvantage is that it is a conservative upper bound and will not be useful unless $p$ is small enough to make $k\cdot p$ relatively small. It is possible for $k\cdot p$ to be greater than 1. There are many other procedures that can be used that are not so conservative and some just involve the individual $p$-values. Examples are bootstrap and permutation adjusted p-values. You can find this in the resampling book for multiple comparisons by Westfall and Young.

$\endgroup$
  • 4
    $\begingroup$ This does not answer the question. It is an useful comment on Bonferroni but should be IMHO (in scope of this question) a comment and not an answer. $\endgroup$ – Erik May 3 '12 at 15:03
  • 1
    $\begingroup$ I find also that our site in particular appears to be remarkably devoid of capricious or malicious behavior. In large part this is a testament to the management of our three outstanding moderators and also to the professionalism of the bulk of our contributors. If you are finding yourself the target of persistent downvoting it may be a sign that one or more community members have reservations about some pattern of use. If you have concerns yourself, I'd suggest either contacting the mods privately or addressing the issue on our meta site. $\endgroup$ – cardinal May 4 '12 at 3:17
  • 2
    $\begingroup$ BTW, I still don't think it answers the question. The original p-values that Bonferroni operates on still come from a t-statistic. $\endgroup$ – Erik May 4 '12 at 6:17
  • 1
    $\begingroup$ @cardinal Is downvoting something that the site encourages if you don't like the tone of an answer rather than just the content? I hope that people here don't use popularity as a criterion for upvoting or downvoting. That was what voting on subscriber book reviews degenerated into on amazon where reviewer get rankings based in favorable and nonfavorable votes. I intend to behave professionally and follow the site rules. Being new (a member for just 3 days now) I am just having a hard time learning and understanding all of them. $\endgroup$ – Michael R. Chernick May 5 '12 at 15:49
  • 2
    $\begingroup$ I did not volunteer that I downvoted, in fact I clearly stated that I did, in fact, not downvote your answer. I had not and still haven't voted on it at all. The tip regarding the software package is good, a screenshot of the settings of the multiple comparison would also help. $\endgroup$ – Erik May 5 '12 at 16:12
1
$\begingroup$

I hope this quote helps you. "As you see in the output below, the table titled "Contrast Coefficients (L' Matrix)" shows the coding scheme that was used for each comparison. The table entitled "Contrast Results (K Matrix)" shows the results of the various contrasts. In our example, the difference between level 1 of race and level 4 of race is statistically significant. You will notice that the contrast estimate is the difference between the mean for the dependent variable for the first level minus the mean of the dependent variable for the omitted level. In other words, the mean for level 1 minus the mean for level 4 which is 46.4583 - 54.0552 = -7.597. The row labeled "Sig." is .000, indicating that this difference is significant, and this is followed by a confidence interval for the difference. The next part of the table compares level 2 of race and level 4 of race and shows that this difference is not statistically significant and the next part of the table shows the difference between level 3 of race and level 4 of race is statistically significant. You might note that while the significance ("Sig.") is given for each of these tests, there is no "t" value, but you could obtain this by dividing the "Contrast Estimate" by the "Std. Error", i.e., -7.597 / 1.989.".

Thus, if you have values for contrast estimate and std erro, you can calculate t value yourself.

$\endgroup$
  • $\begingroup$ Thanks. Where did you get the quote from? $\endgroup$ – Mien Mar 25 '15 at 7:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.