2
$\begingroup$

I have a question concerning the use of the kwallis2 test in Stata. If looking at the multiple comparison output, I receive a probability and (NS) or (S).
What does the probability there mean and to what does the program compare this value to reach the outcome significant or not? I suppose that the standard 5% significance level is used. But I also see some adjusted p-value for significance... I am a little confused about the output given by this test.

Another question: How can the significance level be changed?

I hope (and am thankful) for good explanations!

$\endgroup$
  • 1
    $\begingroup$ The single p-values are compared to the critical value used by the program and reported as Adjusted p-value for significance. If for some reasons you don't like that critical value and, for example, you want to use an $\alpha=0.01$, then you can compare the single p-values for the different comparisons with your $\alpha$ and forget about the (S) and (NS) output. $\endgroup$ – boscovich May 8 '12 at 13:50
  • $\begingroup$ Thank you! But which is the standard significance level used for kwallis2? The adjusted p-value for my test is 0.0041. This is not even close to the 1% significance level. Do you know how the program gets to the adjusted p-value? Thanks again. $\endgroup$ – Katharina Hilken May 8 '12 at 14:28
  • 1
    $\begingroup$ Looking at the code behind kwallis2, in the case of multiple comparisons between groups (i.e. all the possible pairwise comparisons), the critical value $\alpha'$ is calculated as $\alpha/(k*(k-1))$, where $\alpha=0.05$ by default (but can be modified) and $k$ is the number of groups. If you choose a control group and compare all the other groups versus that one, then $\alpha' = \alpha/(2*(k-1))$ $\endgroup$ – boscovich May 8 '12 at 14:34
  • 2
    $\begingroup$ @KatharinaHilken Welcome to the site! Some tips: your posts are automatically signed with your username, so there is no need to do it manually; and if some answer suits your needs, you can "accept it" by pressing green tick mark next to it -- you and the answer author will get a rep bonus and other users can see that your problem was solved. $\endgroup$ – user88 May 8 '12 at 21:10
3
$\begingroup$

The single p-values from the multiple comparisons are compared to the critical value computed by the kwallis2 command and reported as "Adjusted p-value for significance". If for some reasons you don't like that critical value and, for example, you want to use an $\alpha=0.01$, then you can compare the single p-values for the different pairwise comparisons with your $\alpha$ and forget about the (S) and (NS) output.

Looking at the code behind kwallis2, in the case of multiple comparisons between groups (i.e. all the possible pairwise comparisons), the critical value $\alpha'$ is calculated as $\alpha/(k∗(k−1))$, where $\alpha=0.05$ by default (but can be modified using set level before running kwallis2) and $k$ is the number of groups. If you choose a control group and compare all the other groups versus that one, then $\alpha'=\alpha/(2∗(k−1))$

For example:

. sysuse auto, clear
(1978 Automobile Data)

. set level 90 // (i.e. alpha = 0.1)

. kwallis2 length, by(rep78) control(1)

One-way analysis of variance by ranks (Kruskal-Wallis Test)
[omitted output]
(Adjusted p-value for significance is 0.012500)
[omitted output]

. di 0.1/(2*4)
.0125
$\endgroup$
  • 1
    $\begingroup$ if k is the number of groups, the total possible pairwise comparisons would be k*(k-1)/2, so the α ′ should be 2α/(k∗(k−1)). $\endgroup$ – user19694 Jan 15 '13 at 16:58
  • $\begingroup$ Good catch, @Qin. Welcome to our site! $\endgroup$ – whuber Jan 15 '13 at 17:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.