# Friedman's test is very significant, but its post hoc comparisons (SPSS) are not significant

I ran a non-parametric Friedman's test for my data in SPSS 22 and significantly rejected the null. That would mean that among the $k$ paired samples (3 in my case), there should be detected at least two samples with unequal distributions – one tending to be greater than the other. So, post hoc comparisons are justified.

However, if I further run the SPSS built-in post-Friedman post hoc pairwise multiple comparisons, which, according to this SPSS note, are based on Dunn's (1964) approach with the Bonferroni correction, I get non-significance for all the pairs. The omnibus Friedman significance was very persuasive ($p=0.002$), but the results of pairwise post hoc tests are all not significant, even for figures without the Bonferroni adjustment.

Why is it so? Am I doing it wrong or is SPSS?
What is the proper after-Friedman post hoc pairwise testing?

The sample dataset is available here as SPSS data, or as printed next:

V1  V2  V3
5   5   5
4   4   5
5   3   5
4   5   5
5   5   5
5   5   5
5   5   4
5   5   5
5   5   4
5   5   5
5   5   5
4   4   4
4   4   4
4   5   5
3   3   3
4   4   5
3   5   2
5   5   5
3   3   5
4   4   4
5   5   5
5   4   5
5   5   5
5   5   5
4   4   5
5   5   5
5   5   5
5   4   5
5   5   5
5   5   5
4   4   4
4   4   4
5   5   5
4   4   4
4   5   4
5   5   5
4   4   4
4   4   4
4   5   4
5   5   5
5   5   5
5   5   5
5   4   4
5   5   5
4   5   5
5   5   5
5   5   5
5   5   5
5   5   5
4   4   4
5   5   4
5   5   5
5   5   4
5   4   4
5   5   5
4   4   4
4   4   4
5   4   3
5   5   4
4   5   4
5   5   5
5   5   5
4   4   4
5   5   4
5   4   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   4   5
5   5   5
5   5   5
5   4   5
5   5   5
5   5   5
5   5   4
4   4   4
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   4   5
5   5   5
5   5   5
5   5   5
4   4   3
4   4   4
5   5   4
4   4   5
4   5   4
4   3   4
4   4   4
4   4   4
4   4   4
5   4   4
5   4   4
2   2   3
4   4   5
4   4   4
5   4   5
4   4   3
4   4   4
4   4   5
5   2   5
4   3   5
4   4   4
4   5   4
4   4   4
4   5   5
5   5   5
5   5   5
4   5   4
5   3   5
5   5   5
5   4   5
5   3   5
2   3   5
5   5   5
5   5   5
4   4   4
5   5   4
4   5   5
5   5   5
5   5   5
3   4   4
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   4
5   5   5
5   5   5
5   5   3
5   5   3
5   5   5
5   5   3
5   5   4
5   5   3
5   5   3
5   5   5
5   5   5
5   5   3
5   5   4
5   5   3
5   5   5
5   5   3
5   5   5
5   5   3
5   5   4
5   5   5
5   5   5
4   4   4
4   4   4
3   4   4
4   5   5
3   5   4
3   5   4
5   5   5
3   3   4
5   5   5
5   5   5
5   5   4
4   4   4
4   4   4
4   4   4
5   5   5
3   2   4
3   2   4
4   4   5
5   5   5
3   1   2
5   4   1
5   4   5
5   5   5
5   4   3
4   5   4
2   3   5
3   2   1
3   2   2
5   5   5
4   4   5
5   5   1
5   3   3
3   3   4
5   3   4
4   5   5
5   4   3
5   1   4
4   2   2
4   4   2
5   2   1
4   4   5
5   3   5
5   3   5
2   5   4
4   3   4
5   4   4
5   2   1
5   4   2
3   1   5
4   4   5
5   4   2
3   4   1
5   3   2
5   4   5
4   1   5
5   4   5
4   3   5
5   4   5
4   5   5
5   4   4
5   2   2
4   5   4
4   4   5
5   5   3
4   5   4
5   4   4
5   4   4
5   5   5
4   4   4
5   5   5
5   4   3
5   5   5
5   5   5
5   4   5
5   5   5
5   5   5
5   5   5
5   5   5
4   5   5
5   4   4
5   5   5
4   4   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
2   4   5
4   4   4
5   4   4
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
4   4   4
5   5   5
4   5   4
5   4   5
5   5   4
5   4   4
5   5   5
5   2   3
5   2   2
5   2   1
1   1   1
4   4   3
4   4   4
5   4   4
5   5   4
5   4   5
5   4   3
3   5   5
4   3   4
4   3   4
4   4   5
4   4   3
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
4   4   4
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
4   4   4
5   5   5
5   5   5
5   5   5
5   5   5
4   4   4
4   4   4
5   5   5
5   5   4
4   5   5
5   4   4
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
4   4   5
2   4   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
4   4   4
5   5   5
5   5   5
5   4   4
5   4   4
5   5   5
5   5   5
4   5   4
4   4   4
4   3   4
4   4   3
5   4   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   4   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
4   5   5
5   5   5
4   5   4
5   5   5
1   5   4
5   4   5
5   5   5
5   5   5
4   4   4
4   2   5
5   5   5
3   4   5
5   5   5
4   4   4
5   4   4
5   4   5
5   5   5
4   3   4
4   4   4
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5
5   5   5

• To me, it is indeed quite strange. Given the overall test significance as strong as .002 there at least one pair is intuitively expected to differ significantly. By all accounts, it should be V1-V2. Sign test, of which Friedman can be seen as an extension, shows (after values within each respondent, are ranked, as in Friedman test) that pair V1-V2 is highly significant. I'm a bit bewildered and should sit and try to follow SPSS Algorithms doc. – ttnphns Oct 4 '15 at 23:27
• Because you're testing different hypotheses, using different procedures. Apples and oranges. – rvl Oct 5 '15 at 0:41
• @ttnphns pairwise post hoc tests needn't necessarily be "wrong" in any sense, and yet still have none reject where the original test does, since the omnibus test doesn't consist of the smallest pairwise p-value. This property of pairwise tests compared with an omnibus test is explored in all manner of permutations in questions here in CV. e.g. Start with a pairwise procedure and a single pair of groups being compared with it, where the comparison is not quite significant. Call the lower group $A_1$ and the upper group $B_1$. Now construct a large number of similar pairs ...(ctd) – Glen_b Oct 10 '15 at 16:11
• @ttnphns That would partly depend on what exactly you mean by "proper" and "correct" - and that in turn depends on what properties one insists on for the post hoc. The fact that even in the simple case of normal-theory one-way comparisons there are a number of post hoc tests indicates that these considerations are not necessarily obvious. There's a procedure due to Nemenyi that's sometimes used with Friedman (but I don't know that it would necessarily count as proper/correct by all criteria). P.B. Nemenyi (1963) Distribution-free multiple comparisons, PhD thesis, Princeton University – Glen_b Oct 11 '15 at 0:17
• @ttnphns According to Wikipedia (which I've only just now thought to look at), it says this approach is sometimes called the "Nemenyi–Damico–Wolfe–Dunn test" ... so Dunn's name may also be attached to a version of this procedure. – Glen_b Oct 11 '15 at 0:42

SPSS Algorithms state that in doing pairwise comparisons after Friedman test they use the Dunn's (1964) procedure. I didn't read that Dunn's original paper so I can't say if SPSS follows it correctly, - but I've just sat and programmed Friedman's test and its post-hoc pairwise comparisons following the above SPSS algorithms documentation, and I confirm that there is no bug and that my results were identical to what SPSS output and the OP showed in the question. (See my code here).

According to the Dunn's approach (as SPSS carries it out) the test statistic is simply the difference in the mean values of the two samples (variables) being compared, that difference after the values were turned into ranks within cases. (It is the ranks left from Friedman's test computations, that is, ranking of the $k$ [k=3 in our example data] values within each case, with mean rank assignment for ties.) St. error of the statistic is $\sqrt{k(k+1)/(6n)}$. It divides the test statistic to yield standardized statistic $Z$ which is plugged in st. normal distribution to give the (Bonferroni yet uncorrected) 2-sided significance.

This comparison test looks very conservative. It failed to praise the pair V1-V2 as significant: Z=1.838, p=.066 despite that the omnibus Friedman is strongly significant: p=.002. In contrast, Sign test for pair V1-V2 (it will be the same irrespective whether you perform it on the raw values or on the ranks left from Friedman) has Z=3.575, p=.0004.

One reason the SPSS "Dunn's approach" is quite conservative is its st. error formula accounting for all the $k$, not 2, variables.

Another reason why it is so less powerful than the Sign test is that it bases itself on all the $n$ cases, including those with ties, while Sign test discards cases with ties; and there is many cases with ties in our data. The problem of power in conjunction with treatment of ties in tests such as Sign was observed, for example in this Q/A.

I took V1 and V2 and, for cases with ties, untied them in a random fashion (by adding negative or positive noise), and computed Sign test (now based on all $n$ cases of course). 500 such trials gave me mean Z=1.927, which is now far from Z=3.575 and much closer on the road of conservatism towards the observed Dunn's Z=1.838.

I feel myself dissatisfied with SPSS' "Dunn's" pairwise comparisons as they are too conservative/weak. We expect that if an omnibus test is significant post hoc tests will confirm it often, if not ever. In our example, even Bonferroni-uncorrected p-value could not support the omnibus conclusion.

Is SPSS at all correct in adopting the "Dunn's approach" (originally proposed for Kruskal-Wallis; see also this Q/A) for Friedman post-hoc testing? I can't say, being hardly an expert in multiple comparisons. I would encourage somebody who knows it to comment or post a really helpful answer on this thread.

P.S. I'm quite aware that, while Friedman test can be seen as an extension of Sign test from 2 to $k$ samples (variables), a pairwise post hoc test after Friedman is not and should not be exactly the Sign test. Neither it would be Wilcoxon paired-samle test. The "Dunn's approach" (if adapted to paired-sample situation) looks plausible post hoc because it compares, without further ranking, the "horizontal" ranks obtained at Friedman and reflecting all the $k$ samples. What bothered me, though, was that the approach appeared overconservative in the example of the post.

Later Addition. To me, Dunn's approach as it is implemented after Friedman's test in SPSS is incorrect. It does not adjust for ties in the same fashion as the parent omnibus test (Friedman) does it. Actually, it does not adjust for the ties at all, while it should. (The issue of ties handling is touched in the current answer above.)

The formula of Friedman's test statistic (explained in SPSS Algorithms) is $$\chi^2= \frac{[12/(nk(k+1))]\sum^k C^2-3n(k+1)}{1-\Sigma T/[nk(k^2-1)]}$$

The denominator of the formula contains the adjustment for ties. If $k=2$ then quantity $\Sigma T/[nk(k^2-1)]$ is the proportion of cases in which the two variables are equal (tied).

Consider Friedman test performed with our variables V1 and V2 ($k=2$). The proportion of cases with ties is 287/400=.7175 and the test statistic is 13.460, df=1 with significance p=.00024. But the "Dunn's" comparison computed following SPSS formulas will be

Sample1  Sample2  MeanRank1 MeanRank2 TestStat  StError   Z    Sig2side  AdjSig
V1       V2      1.54875   1.45125   .0975     .0500  1.9500  .05118  .05118


Nonsignificant. Why? No proper (Friedman style) adjustment for ties was done.

In the presense of only $k=2$ samples in data a correct post hoc pairwise comparison test must give the same result (statistic and p-value) as the omnibus test - it is actually a property which proves that the post hoc test corresponds (is isomorphic) to the parent omnibus test. It is indeed so with Kruskal-Wallis test and Dunn's test - just program it following SPSS Algorithms and test with V1 and V2 as two independent groups, and you'll get same p=.0153 both for KW and for Dunn. But we saw that a similar equivalence is absent in relations between Friedman test and "Dunn's approach" post-Friedman comparison test.

Conclusion. Post hoc multiple comparison test being performed by SPSS (version 22 and earlier) after Friedman's test is defective. Maybe it is correct when there is no ties, but I don't know. The post hoc test does not treat ties the way Friedman does it (while it should). I cannot say anything about the formula of st. error, sqrt[k*(k+1)/(6n)], they are using: it was derived from discrete uniform distribution, but they didn't write how; is it correct? Either the "Dunn's test approach" was adapted to Friedman inadequatly by SPSS or Dunn's test cannot be adapted to Friedman at all.

• +1, but in general I doubt that the same test can be meaningfully used as a post-hoc to both Kruskal-Wallis and Friedman, given that KW is "unpaired" and Friedman is "paired" (or rather repeated measures / within-subjects). Friedman should be much more powerful than KW. If Dunn ignores this "paired/repeated/within" aspect then is must lose lots of power. (I don't know if it does ignore it.) – amoeba Oct 7 '15 at 12:50
• I would rather agree. Well, perhaps the general idea of Dunn's test could be used as post-hoc Friedman (SPSS didn't forget that it is now paired-sample problem), but makeing it isomorphic in ties treatment to Friedman. I observed in my answer that SPSS seem to have forgotten doing it. – ttnphns Oct 7 '15 at 12:56
• I did not look into the ties issue; there might be an inconsistency there too, but it seems to me that this repeated/unrepeated inconsistency is an additional one. Note that the sign test is paired! – amoeba Oct 7 '15 at 12:58

I did Dunn's test on your data with the dunn.test R package which yielded this:

> library(foreign, pos=14)

> Dataset <- read.spss("/Users/Friedman_Sample.sav", use.value.labels=TRUE,
+   max.value.labels=Inf, to.data.frame=TRUE)

> colnames(Dataset) <- tolower(colnames(Dataset))

> library(relimp, pos=15)

> showData(Dataset, placement='-20+200', font=getRcmdr('logFont'), maxwidth=80,
+   maxheight=30, suppress.X11.warnings=FALSE)

> local({
+   .Responses <- na.omit(with(Dataset, cbind(v1, v2, v3)))
+   cat("\nMedians:\n")
+   print(apply(.Responses, 2, median))
+   friedman.test(.Responses)
+ })

Medians:
v1 v2 v3
5  5  5

Friedman rank sum test

data:  .Responses
Friedman chi-squared = 12.117, df = 2, p-value = 0.002338

> dunn.test(Dataset)
Kruskal-Wallis rank sum test

data: Dataset and group
Kruskal-Wallis chi-squared = 6.8206, df = 2, p-value = 0.03

Comparison of Dataset by group
Col Mean-|
Row Mean |          1          2
---------+----------------------
2 |  -2.399474
|     0.0082
|
3 |  -2.092674   0.306799
|     0.0182     0.3795

• FTF, thanks. But Kruskal-Wallis is irrelavant here, it is independent-samples test. In my view, what is being discussed in this thread, instead, is whether SPSS' adoption of Dunn's logic to pairwise testing is generally right for paired-sample situation, or not. If it is generally ok, then why it gives so strange results and where may SPSS be making a mistake in details? – ttnphns Oct 7 '15 at 17:05
• @ttnphns, but are you sure that SPSS actually "adopts Dunn's logic to paired situation" instead of simply using standard unpaired Dunn's? – amoeba Oct 7 '15 at 17:41
• I thought that you @ttnphns wrote: "It would be nice if somebody else here, using another package, maybe an R user, tests it to compare the results with SPSS' results posted by you. " Am I confused? – FTF Oct 7 '15 at 17:44
• @amoeba, yours is a good comment. As I said at the start of my answer to this thread, I didn't read Dunn's article (or its explanation), so I don't know and haven't tested it if SPSS have bluntly duplicated it onto Friedman (which would turn real facepalm for them) or adopted it to it (with possibly a flaw). I dunno. I was writing my answer instantly after reading the question, and subconsciously have believed from the start that SPSS "is always right", so to speak. So, I don't know. The thread is about what is the correct way, after all. – ttnphns Oct 7 '15 at 17:57
• @FTF, I didn't mean any hint of impoliteness. Again, your answer adds it paint to the picture; thank you. I was only saying that we are considering Friedman paired samples test and possible post hoc test for it. – ttnphns Oct 7 '15 at 18:02

I found (via a ResearchGate question) lots of good stuff in the vignette for the PMCMR package (which is now deprecated in favor of PMCMRplus), including post hoc tests by Nemenyi (1963) and Conover (1999). The vignette (citing Conover,1999) claims that the Quade test is more powerful than the Friedman test if $k<5$, and a post hoc test for that is implemented in these packages too. Its pairwise comparisons seem to give a more satisfying result that is congruent with the omnibus test.

Note also some caveats about the Friedman test quoted below. Following this logic, I've been using regular old Tukey post hocs for a repeated measures ANOVA of rank-transformed data. This takes a bit more doing in terms of R code, but it should be easy in SPSS...just make sure to do the rank transformation on one big vector that pools all repeated measures all at once rather than rank-transforming each measure independently (this caused a problem for a collaborator of mine recently)! Results of this method seem satisfying in Niksr's case too (see below).

## Quote from T. Baguley's weblog, Beware the Friedman test!

The ranks for the Friedman test depend only on the order of scores within each participant – they completely ignore the differences between participants. This differs dramatically from the Wilcoxon test where information about the relative size of differences between participants is preserved. Zimmerman and Zumbo (1993)...explain that the Friedman test...is not really a form of ANOVA but an extension of the sign test...

This is bad news because the sign test tends to have low power relative to the paired $t$ test or Wilcoxon sign rank test. Indeed, the asymptotic relative efficiency relative to ANOVA of the Friedman test is $.955 J/(J+1)$ where $J$ is the number of repeated measures (see Zimmerman & Zumbo, 1993). Thus it is about .72 for $J = 3$ and .76 for $J = 4$, implying quite a big hit in power relative to ANOVA when the assumptions are met. This is a large sample limit, but small samples should also have considerably less power because the sign test and the Friedman test, in effect, throw information away. The additional robustness of the sign test may sometimes justify its application (as it may outperform Wilcoxon for heavy-tailed distributions), but this does not appear to be the case for the Friedman test. Thus, where one-way repeated measures ANOVA is not appropriate, rank transformation followed by ANOVA will provide a more robust test with greater statistical power than the Friedman test.

Sure enough, the rank-transformed RMANOVA does produce a smaller $p$ than the Friedman test in Niksr's case. As for what the appropriate post hoc for a Friedman test is, I'm still wondering myself, so pardon the lack of a definitive answer here, and please comment or edit freely if you can help sort through the choices – it seems there are many. My code below demonstrates the outcomes of the five options in PMCMRplus for Niksr's data using defaults for $p$ value adjustments. Note that defaults differ across tests, which may make them harder to compare. I'm open to suggestions/edits on this too if identical adjustments would be more useful in this answer.

## R code

library(foreign);library(PCMCRplus);library(car);library(lme4);library(multcomp)
quade.test(as.matrix(CVd8a))                                  #this is in the stats package
quadeAllPairsTest(CVd8a)                                      #this requires PMCMRplus

CVd8L=stack(CVd8a);CVd8L\$PID=rep(1:nrow(CVd8a),ncol(CVd8a))   #long format for RMANOVA in R
Anova(lmer(rank(values,'keep')~ind+(1|PID),CVd8L),3,'F')  #1-way RMANOVA, type 3 SS, F test
summary(glht(lmer(rank(values,'keep')~ind+(1|PID),CVd8L),mcp(ind='Tukey')))
cld(lsmeans(lmer(rank(values,'keep')~ind+(1|PID),CVd8L),'ind'))     #compact letter display

# Various Post hocs explicitly intended to follow the Friedman test:
frdAllPairsNemenyiTest(CVd8a)
frdAllPairsConoverTest(CVd8a)
frdAllPairsMillerTest(CVd8a)
frdAllPairsExactTest(CVd8a)


## Output (abridged)

Quade test
Quade F = 6.5769, num df = 2, denom df = 798, p-value = 0.001469

Pairwise comparisons using Quade's test with TDist approximation
V1     V2
V2 0.0034 -
V3 0.0057 0.7832
P value adjustment method: holm

Analysis of Deviance Table (Type III Wald F tests with Kenward-Roger df)
F Df Df.res    Pr(>F)
(Intercept) 1894.7708  1 830.45 < 2.2e-16 ***
ind            6.4579  2 798.00  0.001651 **

Simultaneous Tests for General Linear Hypotheses
Multiple Comparisons of Means: Tukey Contrasts
Linear Hypotheses:
Estimate Std. Error z value Pr(>|z|)
V2 - V1 == 0  -49.311     14.934  -3.302  0.00292 **
V3 - V1 == 0  -43.006     14.934  -2.880  0.01127 *
V3 - V2 == 0    6.305     14.934   0.422  0.90644
(Adjusted p values reported -- single-step method)

ind   lsmean       SE     df lower.CL upper.CL .group
V2  581.9612 14.50235 830.45 553.4957 610.4268  1
V3  588.2662 14.50235 830.45 559.8007 616.7318  1
V1  631.2725 14.50235 830.45 602.8069 659.7381   2
Degrees-of-freedom method: satterthwaite
Results are given on the rank (not the response) scale.
Confidence level used: 0.95
P value adjustment: tukey method for comparing a family of 3 estimates
significance level used: alpha = 0.05

Nemenyi-Wilcoxon-Wilcox all-pairs test for a two-way balanced complete block design
V1   V2
V2 0.16 -
V3 0.24 0.97
P value adjustment method: single-step

Conover's all-pairs test for a two-way balanced complete block design
V1     V2
V2 0.0039 -
V3 0.0141 0.9155
P value adjustment method: single-step

Miller / Bortz et al. / Wike all-pairs test for a two-way balanced complete block design
V1   V2
V2 0.18 -
V3 0.27 0.97
P value adjustment method: none

Siegel-Castellan all-pairs test for a two-way balanced complete block design
V1   V2
V2 0.20 -
V3 0.22 0.82
P value adjustment method: holm

Eisinga, Heskes, Pelzer & Te Grotenhuis all-pairs test with exact p-values for a two-way
balanced complete block design
V1   V2
V2 0.21 -
V3 0.22 0.82
P value adjustment method: holm

• +1. Welcome back (??) ! – amoeba May 9 '18 at 8:33
• I'd like to be if I can find the time! I've missed CV and learned a lot in the meantime, occasionally by just visiting. It's less that I've been gone than just unable to contribute, but I'm grateful that so many others (such as yourself) still do! – Nick Stauner May 10 '18 at 15:41

since the question has passed one year, I'm not sure whether you have settled down this problem. Recently I have met the same confusion that I have a significant result after Friedman test in SPSS but I don't know where the significance from and it seems that the spss were not able to do the post test by Dunnt.

I have checked other resources and statistic information, and my answer is that: first, don't worry about your previous result. the hypothesis in the Friedman is not the hypothesis in the post test; second, the spss could not do the Dunnt post test but we can use Wilcoxon signed-rank, the limitation is that you should pair your samples and use the bofferonie correction to lowdown the type 1 error.

• This is incorrect suggestion - about using Wilcoxon. Friedman cannot be seen as an omnibus extension of it. It is sooner an extension of Sign test. See my answer above. – ttnphns Feb 4 '17 at 7:15
• Oh...I will check your answer more carefully. Thanks.. – Melinna Feb 5 '17 at 7:39
• You commented that "while Friedman test can be seen as an extension of Sign test...the post hoc... Neither it would be Wilcoxon paired-samle test". But it seems that the common way to deal with this problem is using the Wilcon Sign test after you have got a significant result in Friedman. I have learnt this in both this website link and several other papers, see (Hinson, 2003). I'm not arguing you and I'd really hope we can figure it out so can find the best way in using spss but not writing code in R. – Melinna Feb 5 '17 at 8:09
• If my answer is wrong, i would delete it so it won't mislead others.:) – Melinna Feb 5 '17 at 8:10