There is one other question similar to mine (Compare independent numeric vectors (means) in R), but has not been answered (1 year has passed), so I'm repeating the question here.

I have obtained exam scores (0-100) of 200 students in 4 subjects. Two of the subjects are taught using one teaching method, while the other two are taught using another method. The subjects are unrelated to each other (a good score on one does not facilitate a good score in another).

Scores in one of the subjects are almost exclusively in the 50s. The other three subjects have consistently higher scores on average (mid to high 60s).

I wish to see if there is a difference in scores between the two teaching methods.

Initially I thought I could use ANOVA, but then realized that there is no dependent variable here.

What are my options? What is the best way to approach this problem? Thanks in advance.

  • $\begingroup$ What? "Score in the subject" is quite plainly the DV. I think your biggest problem is dealing with inference issues like alternative explanations for the differences than teaching method. $\endgroup$ – Glen_b Aug 17 '14 at 3:44

You have a sample of size $n=200$ (number of students). Each observation is 4-dimensional, and can be denoted for example by

$$\{s^a_{1i},\; s^a_{2i},\; s^b_{3i},\; s^b_{4i}\}$$

where $i$ is the student, $\{a,b\}$ are the two teaching methods, and the numerical subscripts indicate the four subjects taught.

Each observation is considered a realization of a random variable ($S_{ki}$). For a specific $i$, are these four r.v.'s independent? Rather unrealistic to argue so, since they have in common the student itself, who it is reasonable to argue, plays some role in the grades obtained. But we are interested in the teaching method... So under the null hypothesis that the teaching method doesn't matter, we should be able to "swap" teaching methods and have the same distribution, $F_S(s_{ki} \mid a) = F_S(s_{ki} \mid b)$... but we do not posses such data, each subject has been taught under only one teaching method (in other words, your data lacks "control groups").

Consider now an $i$ and a $j$: Can we argue that these two 4-dimensional random vectors are identically distributed? This would imply arguing that all students are identical random variables -which is a standard simplifying assumption.

Under this assumption, you can meaningfully consider sample moments per $k=1,2,3,4$ as estimators of the theoretical counterparts.

So let's say you obtain the four sample means. These are sample means of the conditional distributions -conditional on the teaching method. But also, they are sample means of a different random variable, the subject. Which means that if you find differences, they may be attributed to the teaching method -but they may also be attributed to the peculiarities of the subject itself. Can you argue that the subjects are considered of the same difficulty? Can you argue that the subjects are identically distributed random variables, or at least that they have some basic moments equal? If yes, then you essentially assume that their unconditional mean should be the same.

Given these assumptions (students are identical random variables, subjects are identical random variables), then under the hypothesis that the teaching method "doesn't matter", then all four conditional means calculated from the data should then be equal (probabilistically).

So first, you have to test whether the sample means of subjects can be considered the same, per teaching method.

$$\hat E(S_1\mid a) = \hat E(S_2\mid a) ??$$


$$\hat E(S_3\mid b) = \hat E(S_4\mid b) ??$$

Assume that in both the above cases, you conclude that there is no statistically significant difference.

Can you now proceed and argue that if you test

$$\hat E(S_1\mid a) = \hat E(S_2\mid a) = \hat E(S_3\mid b) = \hat E(S_4\mid b) $$

it will give you a yes/no answer as to whether the teaching method matters?

No. You have not provided evidence that the interaction effect of each teaching method with each subject that it is applied, is identical (per teaching method).

Also, you have not provided evidence that subject $1$ and $2$ are unconditionally identically distributed with subjects $3$ and $4$.

And the problem is, you cannot obtain evidence from your sample for these two issues -because the data you have are given the teaching methods. You can only argue about these two issues, with out-of-sample arguments about what these teaching methods are, whether their approach is such that each interact identically with any subject... and also about what these subjects are, whether there is out-of-sample evidence that they can be considered of the same difficulty (or any other relevant aspect), irrespective of teaching method...

...if you can make these arguments, then you can proceed with this last step. Otherwise, you won't be able to disentangle the effects that may be in operation on the scores.

| cite | improve this answer | |

You do have a dependent variable: scores on an exam (0-100). I would reformat your data so that each participant has four scores, one for each subject. You could then use a mixed linear model to examine the influence of teaching method given individual variation, as well as the variance due to the subject of the exam. Your dataset would look something like this:

Participant Subject Teach.Style Score
1           math    1           100
1           science 1           28
1           english 2           89
1           history 2           36
2           math    1           48
2           science 2           73
2           english 1           23
2           history 2           56
3           math    2           98
3           science 2           28
3           english 1           65
3           history 1           19

IF you were using the statistical programming language R, your code would look something like this:

library(nlme) #if you do not have this package, install it
lme(Score ~ Teach.Style, data = dt, random = ~ 1 | Participant + Subject)

I think your main problem was the structure of the data. I hope this helps you find a solution to a successful analysis and project.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.