After a successful Friedman test, I would like to perform post-hoc analysis using the all pairs comparisons Nemenyi's test. Specifically, as I am interested in finding out if the values of some of my samples are greater than the others, I'm trying to apply a one-tailed Nemenyi's test using the PMCMRplus
R package with the alternative="greater"
parameter. However, it seems to to me that the alternative parameter is always ignored. Example below.
Using the example from the PMCMRplus documentation:
## Sachs, 1997, p. 675
## Six persons (block) received six different diuretics
## (A to F, treatment).
## The responses are the Na-concentration (mval)
## in the urine measured 2 hours after each treatment.
##
y <- matrix(c(
3.88, 5.64, 5.76, 4.25, 5.91, 4.33, 30.58, 30.14, 16.92,
23.19, 26.74, 10.91, 25.24, 33.52, 25.45, 18.85, 20.45,
26.67, 4.44, 7.94, 4.04, 4.4, 4.23, 4.36, 29.41, 30.72,
32.92, 28.23, 23.35, 12, 38.87, 33.12, 39.15, 28.06, 38.23, 26.65),nrow=6, ncol=6,
dimnames=list(1:6, LETTERS[1:6]))
friedmanTest(y)
I obtain a p-value less than 0.05
Friedman rank sum test
data: y
Friedman chi-squared = 23.333, df = 5, p-value = 0.0002915
So my understanding is that I can apply post-hoc analysis. I would like to use the Nemenyi's test to perform all possible pairwise comparisons:
## Nemenyi's test
frdAllPairsNemenyiTest(y)
Results:
Pairwise comparisons using Nemenyi-Wilcoxon-Wilcox all-pairs test for a two-way balanced complete block design
data: y
A B C D E
B 0.1880 - - - -
C 0.0917 0.9996 - - -
D 0.9996 0.3388 0.1880 - -
E 0.0395 0.9898 0.9996 0.0917 -
F 0.0016 0.6363 0.8200 0.0052 0.9400
P value adjustment method: single-step
However, in my case I have to perform a one-tailed test as I want to compare if some samples have greater values than others. Hence:
## Nemenyi's test
frdAllPairsNemenyiTest(y, alternative="greater")
But I get the exact same results as before:
Pairwise comparisons using Nemenyi-Wilcoxon-Wilcox all-pairs test for a two-way balanced complete block design
data: y
A B C D E
B 0.1880 - - - -
C 0.0917 0.9996 - - -
D 0.9996 0.3388 0.1880 - -
E 0.0395 0.9898 0.9996 0.0917 -
F 0.0016 0.6363 0.8200 0.0052 0.9400
P value adjustment method: single-step
And the same happens if I try :
frdAllPairsNemenyiTest(y, alternative="less")
Results:
Pairwise comparisons using Nemenyi-Wilcoxon-Wilcox all-pairs test for a two-way balanced complete block design
data: y
A B C D E
B 0.1880 - - - -
C 0.0917 0.9996 - - -
D 0.9996 0.3388 0.1880 - -
E 0.0395 0.9898 0.9996 0.0917 -
F 0.0016 0.6363 0.8200 0.0052 0.9400
P value adjustment method: single-step
This leads me to believe that the alternative
parameter does not work as intended. Am I misinterpreting the results? or am I missing something about the test?
EDIT: the same package offers a frdManyOneNemenyiTest
function for many-to-one comparisons using the same test. When using that function the alternative parameter is taken into consideration:
frdManyOneNemenyiTest(y, alternative='greater')
frdManyOneNemenyiTest(y, alternative='less')
frdManyOneNemenyiTest(y)
Results:
> frdManyOneNemenyiTest(y, alternative='greater')
Pairwise comparisons using Nemenyi-Wilcoxon-Wilcox-Miller many-to-one test for a two-way balanced complete block design
data: y
A
B 0.04119
C 0.01845
D 0.72552
E 0.00747
F 0.00027
P value adjustment method: single-step
alternative hypothesis: greater
> frdManyOneNemenyiTest(y, alternative='less')
Pairwise comparisons using Nemenyi-Wilcoxon-Wilcox-Miller many-to-one test for a two-way balanced complete block design
data: y
A
B 1.00
C 1.00
D 0.91
E 1.00
F 1.00
P value adjustment method: single-step
alternative hypothesis: less
> frdManyOneNemenyiTest(y)
Pairwise comparisons using Nemenyi-Wilcoxon-Wilcox-Miller many-to-one test for a two-way balanced complete block design
data: y
A
B 0.08243
C 0.03711
D 0.99817
E 0.01475
F 0.00052
P value adjustment method: single-step
alternative hypothesis: two.sided
```
alternative
is an argument of the function (rather the value of alternative is returned along with other values): rdocumentation.org/packages/PMCMRplus/versions/1.7.1/topics/… I'm not familiar with this test but if the sampling distribution is symmetrical you can just divide the p-value by 2 to get the one-tailed test (provided the sample effect is in the hypothesized direction) $\endgroup$frdManyOneNemenyiTest
for many to one comparisons that does take into account the alternative parameter (frdManyOneNemenyiTest(y, alternative='greater')
andfrdManyOneNemenyiTest(y, alternative='less')
do give me opposite results as expected) Perhaps I can just use this function in a round-robin fashion to compare all my samples? $\endgroup$