Why is it valid to account for k-1 intercepts w/ only 1 random intercept parameter?

Question

I possess a basic understanding of random vs. fixed effects, and how to code random effects models in SAS. However, I'm having trouble wrapping my head around the derivation of random effects terms, and how a random intercept model, for example, can describe the variation in $k$ intercepts with a single parameter ($\sigma^2$ for a normal dbn) rather than $k-1$ parameters, which can amount to a huge saving in degrees of freedom. Isn't that cheating? ;)

The old school technique would be to use the maximum likelihood method to solve for $k-1$ parameters for each category.

• How does a random intercept model avoid this by using a single parameter?
• Isn't the end result the same--both random and fixed effect models will estimate $k-1$ intercept terms?

Answer 1

Things are a little more complicated with mixed effects models (as you are realizing).

The estimated random effects from the normal model are not exactly the same as the fixed effects that you would compute if you calculated each of the individual intercepts. The fixed effect model assumes that all the groups have the same variance, but each has its own mean and the means are computed independently of each other (basically the mean/intercept for each group). In the mixed effects model the assumption that the random intercepts come from a normal distribution allows the methods to "borrow" information from all the groups in calculating each intercept so the individual intercepts tend to be "shrunken" towards the overall mean, an effect that is often referred to as "regression to the mean".

The estimated random effects are not really parameters, but estimates of random variables, so they don't cost the same number of degrees of freedom. But you are correct in being concerned that there should only be a cost of 1 degree of freedom, the true cost is probably somewhere between $1$ and $k-1$. Here is a post that gives some more detail and an additional reference.

Of course discussing the degrees of freedom assumes that the ratios follow an F distribution, which is itself questionable in mixed effects models.

Answer 2

as ive studied the random vs fixed effecs i neither could get the thing where u "estimate" parameters without loosing df. but the big difference as already mentioned is that the estimating of RE is not really a estimation in the classical OLS sense. in my understand it is better to call it "prediction" since we estimate the outcome of a random variable rather then of a fixed parameter of the given universe.

in case you can read stuff in german i could recommend you a book if you like.