# Understanding the effect of a continuous random factor in a mixed effects model

I understand the effect of a categorical random effect on a mixed effects model in that it performs a partial pooling of the observations by level in the random effect, effectively assuming that the observations aren't independent themselves but only their partial pools are. Also to my understanding, in such a model observations sharing the same random effect level but differing in their fixed effect level will outweigh observations differing in both their random effect and fixed effect levels.

What is the effect of a continuous random factor then? Given that a model without the random effect showed that the fixed effect had an effect size X. Should I expect that if the observations in the different levels of the fixed effect came from far ends of the random effect continuum the effect size will become smaller in a model which included the random factor, while if observations in different fixed factor levels had similar random effect values then the effect size would increase?

• Can you supply the formulae, and/or R/Stata code to exemplify your thinking? You are using a somewhat unusual language... at least unusual to me. I think that your "continuous random factor" is what I would call "random slope", but I first wanted to check. – StasK Jun 14 '13 at 14:59
• @StasK In R terms: if the random factor is categorical (factor in R) then the observations are partially pooled, i.e the group means (random factor levels) are weighted averages of the population mean and the un-pooled group means with weights proportional to the sample size and the inverse of the variance. My question is, what is being done when the random factor is continuous (numeric in R terms). How does that affect the model? – Roey Angel Jun 16 '13 at 9:27
• @RoeyAngel: probably it does not affect it in any sensible way. Specifically for R's lmer for example a model where the random effect has a distinct value for each data-point will fail to even compute. Think of it in purely conceptual terms: if your $Z$ matrix is square then you $\gamma$ vector holding the random effects realization will be of size $N$ ($N$: # of sample points) and thus you'll have an unidentifiable error structure. Are you sure you are asking this? As StasK, I also find it a bit hard to follow your question. – usεr11852 Jun 18 '13 at 3:45
• @user11852 hmmm I honestly never tried it myself with a random effect where each point has a unique value. So what you're basically saying is that a random effect is always treated as a categorical factor (i.e. there's no parallel to how continuous vars are treated in an ANCOVA for example). – Roey Angel Jun 18 '13 at 11:51
• @RoeyAngle: I don't know about ANCOVA specifically, but certainly what I said about non-identifiability stands. You can't estimate $\gamma$ if the $\gamma$ equals the size of your data. It has be treated as categorical as $Z$ reflects a structure (ie. categorization) of the data themselves (eg. batch, group, location etc.). Think of it in the context of hierarchical models (a subset of mixed models): if a hierarchy defined at some level as many descendants as data points then it would be redundant. – usεr11852 Jun 18 '13 at 12:16

I had to think hard about what you were asking. At first I thought along the lines of @user11852, that you were wanting every observation to have its own unique random effect. That would make the model hopelessly unidentified, as there would be no conceivable way to distinguish random effect variation from the model error.

But I believe that in the scope of your intended question, all random effects are actually continuous, and probably normally distributed. However, your allusion to "categorical" is not off the wall, because the design matrix for a random intercept (typically called Z) would look like a design matrix for a categorical variable.

Let's add a bit of concreteness and say that the linear predictor is $$(\bar{\alpha} + \alpha_i) + (\bar{\beta} + \beta_i) x_{ij},$$ where $\bar{\alpha}$ and $\bar{\beta}$ are the fixed effects and $\alpha_i$ and $\beta_i$ are the $i$-specific random effects. I think that by "continuous," you mean a random effect like $\beta_i$ rather than $\alpha_i$. Note that both of these are still constant within a subject $i$.

Now let's think of your proposed situation:

different levels of the fixed effect came from far ends of the random effect continuum

If we consider $\bar{\beta}$ to be the fixed effect, then it couldn't have different levels, but $x_{ij}$ could. Let's assume that for small values of $x_{ij}$, the slope is smaller; $\beta_i$ is negative for subjects $i$ with mostly small values of $x_{ij}$. Now by construction, the extremes of the $x_{ij}$ correspond to the extremes in $\beta_i$.

That leaves us with what happens with vs without the random effect. My thoughts are, if there were only a few extreme cases of the situation above, adding a random effect would tend to pull the estimate of $\beta$ upwards. But I'm not totally sure. In traditional linear mixed modeling, the estimates of the fixed effects are really just weighted least squares estimates. While those weights are directly related to the random effects distribution, their impact will diminish as your sample size increases. In a realistic setting with even moderate sample sizes, I wouldn't expect anything too extreme to happen to your fixed effect estimates when you add in a random effect.