# Test of independence with three levels on 6 observations on a 1-7 Likert rating

I've data from an in between subject study asking participants to rate features in respect to 3 objects, let's say a plant, a stone and a bottle. Each participant gets shown one object and subsequently is asked to attribute a set of 6 features (like "Organic", "Spontaneous", "Complex", ... "Spiritless" - for simplicity here 'a' to 'f') on a Likert scale ranging from 1 "not at all" to 7 "Very much". First participant rates the set of features to the plant, second participant to the stone, third to the bottle, fourth to the plant again and so on. This results in the following data:

#for the plant(20 participants):
a.1 <- c(1,3,1,1,5,5,2,1,7,2,4,2,5,3,5,1,5,3,1,1)
b.1 <- c(1,5,6,7,6,2,1,4,7,1,6,4,7,6,1,2,6,4,5,6)
c.1 <- c(4,6,5,6,6,5,7,7,1,3,7,7,4,5,5,7,5,4,7,5)
d.1 <- c(4,3,5,5,7,7,6,7,7,5,6,5,7,4,5,2,6,5,6,5)
e.1 <- c(5,5,6,4,4,3,4,4,4,1,7,5,3,3,5,5,6,3,4,4)
f.1 <- c(3,4,1,1,4,1,1,4,4,1,1,2,5,4,1,5,3,5,2,5)

#for the stone(19 participants):
a.2 <- c(1,4,1,4,4,2,1,2,2,1,3,3,1,5,2,4,3,1,2)
b.2 <- c(4,5,6,2,3,5,4,4,3,4,3,5,6,3,4,5,4,3,6)
c.2 <- c(5,1,5,7,7,4,5,5,6,5,5,4,3,5,2,5,4,5,3)
d.2 <- c(7,2,4,4,4,4,5,4,4,1,3,2,3,5,4,4,5,4,4)
e.2 <- c(5,1,3,2,5,4,6,3,4,3,3,2,1,3,4,1,3,2,2)
f.2 <- c(4,1,7,1,1,2,3,3,4,5,3,3,7,6,5,6,3,5,6)

#for the bottle(18 participants):
a.3 <- c(1,1,6,3,6,1,5,5,1,2,5,2,3,3,6,5,7,4)
b.3 <- c(5,5,6,3,6,4,3,4,6,4,5,4,4,5,4,5,3,4)
c.3 <- c(3,4,4,2,4,7,5,4,7,1,3,6,5,5,3,4,3,5)
d.3 <- c(3,2,6,3,5,3,3,4,4,6,4,4,4,6,4,2,2,3)
e.3 <- c(1,6,5,1,3,1,3,4,1,2,4,7,3,5,3,3,2,2)
f.3 <- c(4,3,5,5,2,4,5,4,2,5,5,1,2,3,2,5,6,4)


Subsequently the data is transformed into a 6x7 contingency table representing the count or frequency measures of each of the feature's ratings for each object:

df.flower <- t(apply(rbind(a.1,b.1,c.1,d.1,e.1,f.1),1,tabulate, nbins=7))
df.stone <- t(apply(rbind(a.2,b.2,c.2,d.2,e.2,f.2),1,tabulate, nbins=7))
df.bottle <- t(apply(rbind(a.3,b.3,c.3,d.3,e.3,f.3),1,tabulate, nbins=7))

> as.data.frame(df.flower)
V1 V2 V3 V4 V5 V6 V7
a.1  7  3  3  1  5  0  1
b.1  4  2  0  3  2  6  3
c.1  1  0  1  3  6  3  6
d.1  0  1  1  2  7  4  5
e.1  1  0  4  7  5  2  1
f.1  7  2  2  5  4  0  0

> as.data.frame(df.stone)
V1 V2 V3 V4 V5 V6 V7
a.2  6  5  3  4  1  0  0
b.2  0  1  5  6  4  3  0
c.2  1  1  2  3  9  1  2
d.2  1  2  2 10  3  0  1
e.2  3  4  6  3  2  1  0
f.2  3  1  5  2  3  3  2

> as.data.frame(df.bottle)
V1 V2 V3 V4 V5 V6 V7
a.3  4  2  3  1  4  3  1
b.3  0  0  3  7  5  3  0
c.3  1  1  4  5  4  1  2
d.3  0  3  5  6  1  3  0
e.3  4  3  5  2  2  1  1
f.3  1  4  2  4  6  1  0


I then apply a chi-square test to check whether there's significant difference between the features and the rating-frequencies for each of the objects:

chisq.test(df.flower)
chisq.test(df.stone)
chisq.test(df.bottle)


Ultimately, my question and intend is to find a possible method to determine the (significance of the) difference between the ratings of the three objects.

Thus, in an ideal world I would like to be able to conclude something along these lines: participants ratings of the feature-set in respect to the flower are significantly different to the ratings for the stone and the bottle. However the ratings for the stone and the bottle are not significantly different. (This result is made up just to illustrate the point and not based on any calculation)

• Questions about the appropriateness of various statistical tests are off-topic for Stack Overflow. You should ask such questions at Cross Validated where questions about statistical topics are welcome. – MrFlick Feb 14 '17 at 23:47
• While asking for R implementation is off-topic, the question asks for a suitable test, so sounds on-topic to me. – Firebug Feb 15 '17 at 13:45
• Are you asking what to do with a 3 by 2 by 7 contingency table? – mdewey Feb 16 '17 at 16:13
• Thanks for moving this on. My wording might be a bit byte confusing and I aim to change the description above asa I know how to put it better. For the time being I made an edit that hopefully contributes to clarify the question. – raumkundschafter Feb 16 '17 at 19:01
• I don't see a contingency table as being appropriate for this. I am not even sure what the data are supposed to represent. Are your observations (i.e. your rows) repeated measurements on a person, such that each person has 3 units (which you refer to as levels)? If not, what are the levels? You can use something like Spearman's correlation, which is rank-based, to get a sense of how the ratings correlate among the observations, if that's what you want (actually, because your data are already ordinal, Spearman may = Pearson in this case, but don't quote me on that). – Weiwen Ng Feb 17 '17 at 14:44

The Dunn test seem to provide at least partial results. When executed on each feature (a to f) in respect to the objects, significance in the difference can be determined e.g.:

df.flower <- as.data.frame(t(rbind(a.1,b.1,c.1,d.1,e.1,f.1)))
names(df.flower) <- c("a","b","c","d","e","f")
df.flower$Type <- as.factor(rep("Flower", nrow(df.flower))) df.stone <- as.data.frame(t(rbind(a.2,b.2,c.2,d.2,e.2,f.2))) names(df.stone) <- c("a","b","c","d","e","f") df.stone$Type <- as.factor(rep("Stone", nrow(df.stone)))
df.bottle <- as.data.frame(t(rbind(a.3,b.3,c.3,d.3,e.3,f.3)))
names(df.bottle) <- c("a","b","c","d","e","f")
df.bottle\$Type <- as.factor(rep("Bottle", nrow(df.bottle)))

df.bind <- rbind(df.flower,df.stone,df.bottle)

library(FSA)
PT.a = dunnTest(a ~ Type, data=df.bind, method="bh", two.sided=TRUE)
PT.a
print(PT.a,dunn.test.results=TRUE)
PT.d = dunnTest(d ~ Type, data=df.bind, method="bh", two.sided=FALSE)
PT.d
print(PT.d,dunn.test.results=TRUE)


for the latter (feature d) the chi-square result is 14.5658, df = 2, p-value = 0 and in the post-hoc test splits up as follows:

       Comparison          Z      P.unadj       P.adj
1 Bottle - Flower -3.3872312 0.0007060187 0.002118056
2  Bottle - Stone -0.2587276 0.7958453871 0.795845387
3  Flower - Stone  3.1695055 0.0015269857 0.002290478


Thus indicates there's a significant difference between the Bottle & the Flower and the Flower & the Stone in respect to feature d. Hence applying the Dunn test allows to determine 'driving features' from the set. However applying it to all the data results in a rather excessive results list. Factor analysis might help to reduce this but guess this would be going into another thread.