# Can statistics really be used in education to determine test scores?

I am a middle school teacher. My district prides itself on having the kids take 3 tests during the year to predict their score, and my teaching, on "the real test" at the end of the year. My question: Is this practice even real? Or is the district fooling themselves in thinking these numbers mean more than what they do?

-
What do the data show? – whuber Mar 14 '14 at 18:33

Probably some combination of both.

• Have the (predictive) tests been validated as measuring the same constructs as "the real test"?
• Are there three different tests (types) or is it the same test three times (tokens)? If it is the same test three times, the students may get better at it by virtue of exposure, rather than by having learned anything more. The question then is whether that fact had been taken into account when modeling the predictive relationship.
• In addition, what happens if scores are low to these tests? If the school responds by helping and/or providing additional resources during the remainder of the year, that can change the predictive relationship between the predictive tests and the real test. For example, the predictive relationship may have been modeled as what would have happened if nothing had changed, but it could also be modeled as what is likely to happen given the changes that will be made as a result.
• Moreover, these tests will always have some measurement error, so the scores will bounce around; unless the measurement error is very low, the prediction can only be so good w/ just three measurements per student.
-
I'm guessing that #2 is more of feature than a bug here, at least as far as the individual school is concerned. – Matt Krause Mar 14 '14 at 19:40
@MattKrause, if the predictive tests are the same test as the 'real' test, probably so. – gung Mar 14 '14 at 19:49
Even if the individual questions are different across sessions, there's some benefit to getting familiar with the format and the type of questions. If you take the SAT twice, you're likely to better the second time around. In theory, one could model this out, but I'd be surprised if anyone actually does.... – Matt Krause Mar 15 '14 at 3:57

@tom: Not sure what you mean by "real", but if it is happening in the real world, it appears that it is real enough. The question is perhaps one of validity, and I would encourage you to search this site for questions and answers on reliability and validity. This is a very, very large field. If you wish to discuss this in any depth, it would be good to become familiar with the loaded terms that are used.

Basically, the question rests on the quality of "the real test" at the end of the year and the quality of the classroom tests during the year. The school district should have evidence upon which to base its actions, so I sincerely hope you could retrieve that evidence by contacting the district.

If the evidence does not exist, then this is a research project in the making. School and district decisions should be evidence based. Again, this is a huge field, please have a look at this site in more depth.

-

It is certainly possible that practice test scores can predict (not determine) test scores. However, this depends a lot on how well the practice tests simulate the real test. For instance, practice ACT/SAT scores could certainly be used to predict how a student may do on the actual ACT/SAT. On the other hand, giving a middle school English class a practice test from an AP physics book is certainly not a valid test to measure English ability, and it would likely not predict their final test results very well at all.

-
Yes, but it's worth noting that tests always tap more constructs beyond just what they are intended to measure. Eg, an English test (TOEFL) can help predict physics GRE scores for non-native English speakers. – gung Mar 14 '14 at 19:18