2
$\begingroup$

I was having a discussion with a colleague yesterday about an analysis he was doing with some student achievement data. We got into a discussion of value added models (VAM), which in my understanding are just multilevel models controlling for a prior assessment occasion. The analyst had the idea of using the random effects estimates from one model as input for a structural equation model with the same sample using a different set of data. My initial thought was: That doesn't sound copasetic! However, I had a hard time articulating why. My thoughts were that the random effects exhibit different degrees of variability, and to use them as point estimates in another model kind of misses the point of examining them in the first place. The other thought I had was in my experience with multilevel modeling, the random effects are so contingent on aspects of the model (i.e., covariance structure, variables specified, etc.) that I wouldn't dare treat the random effects as a "reality."

Does anyone else have experience with people who have thought of using random effects as independent variables in another model? If others agree that this is not best-practice, then are there some more technical pointers I can give my colleague about why?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ This doesn't answer your question, but I think it is important to mention that many mathematicians and statisticians argue that the value added model (at least as it is deployed) basically equates to evaluating a teacher based on random chance (the data is extremely noisy and the per teacher sample sizes are too small). garyrubinstein.teachforus.org/2012/02/28/… $\endgroup$
    – WetlabStudent
    Apr 13, 2015 at 16:52
  • 1
    $\begingroup$ I totally agree with you! I'm not saying I agree with what the person is trying to do. I think we start going down a dangerous road when we accept these kinds of models as summitative vs formative metrics. $\endgroup$
    – bfoste01
    Apr 13, 2015 at 18:04

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.