Here are data from a tasting, conducted by a newspaper, of seven brands of chocolate pudding, by five different tasters. As in your Question, each
taster rated each brand, resulting in 35 scores. In this case, each score
was said to be the sum of several partial scores, so there is some reason to believe scores may be normal.
x = c(7,12,18,10,11,6,5, 15,14,8,6,2,0,6,
20,10,20,4,13,0,7, 18,16,12,10,8,9,3,
15,16,9,13,3,5,5)
matrix(x, byrow=T, nrow=5)
[,1] [,2] [,3] [,4] [,5] [,6] [,7]
[1,] 7 12 18 10 11 6 5
[2,] 15 14 8 6 2 0 6
[3,] 20 10 20 4 13 0 7
[4,] 18 16 12 10 8 9 3
[5,] 15 16 9 13 3 5 5
This amounts to a two-way ANOVA with one replication in each of 35 cells (so
that there can be no interaction term). Looking at brands of interest
as a fixed effect and tasters as randomly chosen, it might be considered
a complete block design. (Due to possible quirky taste preferences of
various tasters, there may be interaction. But with one observation
per cell, we have no way to test for interaction.)
An ANOVA procedure in R finds that there are significant differences
among brands but not significant differences among tasters.
If you look at both effects as fixed, the ANOVA model would be:
$$Y_{ij} = \mu + \beta_i + \tau_j + e_{ij},$$
for $i=1,\dots,7$ brands, $j = 1,\dots,5$ tasters and
$e_{ij} \stackrel{iid}{\sim}\mathsf{Norm}(\mu=0,\sigma),$ with
$\sigma^2$ to be estimated by MeanSq(Resid).
brand=as.factor(rep(1:7, times=5))
tastr=as.factor(rep(1:5, each=7))
anova(lm(x ~ brand + tastr))
Analysis of Variance Table
Response: x
Df Sum Sq Mean Sq F value Pr(>F)
brand 6 580.80 96.800 5.8113 0.000748 ***
tastr 4 55.83 13.957 0.8379 0.514653
Residuals 24 399.77 16.657
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
With appropriate protection against 'false discovery' (e.g., Bonferroni)
paired t tests might reveal which brands differ in significant and
important ways.
You can look at various help pages online to see how to test whether
residuals from the ANOVA model are nearly normally distributed.
If you feel that residuals are too far from normal, you might use a nonparametric Friedman test to look for differences among brands.
friedman.test(x, brand, tastr)
Friedman rank sum test
data: x, brand and tastr
Friedman chi-squared = 19.321, df = 6, p-value = 0.003654
Paired Wilcoxon (signed rank) tests might be used ad hoc to explore differences among brands.
