Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I am interested in the way in which data are analysed in education. In England, pupils' reported test results are compared to national test results using z-tests. These results are then used to judge the effectiveness of schools. My hypothesis is that students in a school are not IID, and that therefore analyses using tests based on the Central Limit Theorem which seeks to judge a 'school effect' rather than a 'pupil effect' is not justified.

Is it reasonable to assume that students and therefore their test scores are independent and identically distributed?

share|improve this question
    
When people use the phrase "independent and identically distributed" they're talking about random variables being independent of each other, and identically distributed to each other. You seem to mean something else. In which case, what's the random variable or variables, independent of what, and identically distributed to what... and against which alternatives (what kind of dependence, for example)? –  Glen_b Jul 15 at 0:22
    
Thanks Glen_b. To be clear, I hypothesise that pupils are not randomly distributed between schools - that they are, to a large extent, dependent on each other within schools, and that they are, also to a large extent, not identically distributed between schools, with clearly different types of pupils attending different schools. My suggestion is that children in a given school have similar backgrounds, levels of parental wealth, educational attainment and aspiration and so on, so children in different schools cannot simply be interchanged as they are not i.i.d. –  Jack Marwood Jul 15 at 11:12
    
I may be using i.i.d. in an unconventional or incorrect way: It would be useful to get some feedback on this. It is worth noting that it can be clearly demonstrated that pupils have differing levels of test scores on average dependent on their family socio-economic status. I don't think that this, however, would allow you to directly link an individual's SES with their test score (since there is naturally some variation at an individual level). –  Jack Marwood Jul 15 at 11:12
    
That is why I asked "Is it reasonable to assume that students and therefore their test scores are independent and identically distributed?" I hope this clarifies my original question. –  Jack Marwood Jul 15 at 11:13

1 Answer 1

One approach to problems like this is "multilevel" or "hierarchical" modeling: build a model for the individual students' scores which are nested within a model for the schools.

The problem you pose is actually very similar to the classic Bayesian problem for "the eight schools," which is discussed in Gelman's Bayesian Data Analysis: students in eight schools are coached on how to improve their test scores. The posterior distribution of effect sizes for the coaching for all eight schools are shrunk towards zero, whereas the non-hierarchical result (corresponding to no pooling across schools) provides the opposite results, partially owing to its unrealistic framing of the problem.

So, no, within the same school, it is unlikely that the students are i.i.d.-- the students likely have teachers in common and so forth, which creates some dependence among students. We can imagine that a class of students would not do well on an elementary math test if their teacher decided to skip the lessons on fractions, but otherwise covered the rest of the material. So you would have to build a model accounting for the dependence among these student-level observations.

Across the entire population of students, the same form of dependence must exist: if students in the same school or class are more similar than students in other schools or classes when looking at just two or three schools or classes, then this effect must be present in the whole nation, when you are looking at all of the schools, or all of the classes.

However, it's less clear whether this dependence is important in the particular context of some specific research question. This is because the effect of dependence on quantities of interest may be quite small, or because specific research designs mitigate this dependence.

share|improve this answer
    
Thanks for this detailed reply, and I look forward to following up your comments re Gelman's work. I am, however, actually interested in the idea of whether pupils across the population can be assumed to be i.i.d., not whether pupils are simply i.i.d. within a school. As Data is often assumed to represent a 'school effect' based on CLT analysis, this basic assumption is clearly important. I hope that this clarifies my query. –  Jack Marwood Jul 14 at 17:05
1  
If I understand correctly, there would still exist the same kind of dependence: as long as students in classes are also members of the national population of students, the same kind of dependence appears. Whether this is substantively interesting or important for some particular line of analysis is unclear. –  user777 Jul 14 at 17:31

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.