Chi-squared vs ANOVA test My study consists of three treatments. One treatment group has 8 people and the other two 11.
Each person in the treatment group received three questions and I want to compare how many they answered correctly with the other two groups. 
How would I do that? Should I calculate the percentage of people that got each question correctly and then do an analysis of variance (ANOVA)? Somehow that doesn't make sense to me.
 A: Since your response is ordinal, doing any ANOVA or chi-squared test will lose the trend of the outputs. Consider doing a Cumulative Logit Model where multiple logits are formed of cumulative probabilities.
$$
\begin{align}
P(Y \le j | x) &= \pi_1(x) + ... +\pi_j(x), \quad j=1, ..., J\\
logit\big[P(Y \le j | x)\big] &= \frac{P(Y \le j | x)}{1-P(Y \le j | x)}\\
&= \frac{\pi_1(x) + ... +\pi_j(x)}{\pi_{j+1}(x) + ... +\pi_J(x)}
\end{align}
$$
Finally we assume the same effect $\beta$ for all models and and look at proportional odds in a single model.
$$
logit\big[P(Y \le j |\textbf{x})\big] = \alpha_j + \beta^T\textbf{x}, \quad j=1,...,J-1
$$
In this case, you would have a reference group and two $x$'s that represent the two other groups
$$
logit\big[P(Y \le j |\textbf{x})\big] = \alpha_j + \beta_1x_1 + \beta_2x_2
$$
Model fit is checked by a "Score Test" and should be outputted by your software. The Score test checks against more complicated models for a better fit.
Finally, interpreting the results is straight forward by moving the logit to the other side
$$
P(Y \le j |\textbf{x}) = \frac{e^{\alpha_j + \beta^T\textbf{x}}}{1+e^{\alpha_j + \beta^T\textbf{x}}}
$$
There are lots of more references on the internet. Agresti's Categorial Data Analysis is a great book for this which contain many alteratives if the this model doesn't fit.
If you want to stay simpler, consider doing a Kruskal-Wallis test, which is a non-parametric version of ANOVA. Like most non-parametric tests, it uses ranks instead of actual values and is not exact if there are ties. Like ANOVA, it will compare all three groups together. While it doesn't require the data to be normally distributed, it does require the data to have approximately the same shape. 
If the null hypothesis test is rejected, then Dunn's test will help figure out which pairs of groups are different. It is also based on ranks,
A: I don't think you should use ANOVA because the normality is not satisfied. Furthermore, your dependent variable is not continuous.
Mann-Whitney U test will give you what you want. Your dependent variable can be ordered (ordinal scale). Say, if your first group performs much better than the other group, you might have something like this:
1 1 1 1 1 2 1 1 2 2 2 2
The samples are ranked according to the number of questions answered correctly. In this example, group 1 answers much better than group 2.
