3
$\begingroup$

Sometime I hear people talk about whether to take a variable both as fixed and random effect or not (e.g., this blogpost) in the context of frequentist mixed-effects models.

According to this part of the blogpost, a grouping variable is also considered a random-effect. Therefore, the discussion regarding whether to take a variable both as fixed and random effect or not has been represented for instance by the following syntax (using R's lme4 package):

lmer(Y ~ X + (1|X), data = data)   ## DON'T RUN (take X both as fixed and random)

# OR

lmer(Y ~ 1 + (1|X), data = data)   ## DON'T RUN (take X only as random)

However, I know we actually take the intercepts or slopes (or both) of a fixed-effect variable (e.g., A) to vary (i.e., be random) among each level of a grouping variable (X).

Question: So, shouldn't the the discussion regarding whether to take a variable both as fixed and random effect or not be represented for instance by the following syntax:

lmer(Y ~ A + (A|X), data = data)   ## DON'T RUN (take A both as fixed and random)

# OR

lmer(Y ~ A + (1|X), data = data)   ## DON'T RUN (take A only as fixed)

My question above is also supported by this post on CV.

TL;DR:

When we say we are unsure whether to take a variable both as fixed and random or not; do we mean THIS SITUATION or THIS OTHER SITUATION (just focus on their model syntaxes)?

$\endgroup$
3
  • 1
    $\begingroup$ (I) I think this borders on being an XY Problem, because there are two issues at play here. (1) What does "random effect" refer to? The ambiguity in this terminology is well established -- without context, it's not clear if it's the BLUPs, the grouping variable, or the combined "term" in the model. (2) For which, if any, senses of "random effect" does it make sense to have a random effect also be a fixed effect. $\endgroup$ – Livius Dec 11 '20 at 14:53
  • 1
    $\begingroup$ (II) For (1), this is a well known problem and even applies to the terminology "fixed effect", which for example means something like "categorical variable" in econometrics. Gelman and Hill (2006) actually spend half a page discussing this terminological issue. In many cases, the ambiguity is resolved by context, but can often be avoided by using e.g. "variance components" or "varying intercepts/slopes by group". For (2), there are lots of answers across CV, usually broken up by the sense, e.g. this. $\endgroup$ – Livius Dec 11 '20 at 14:56
  • $\begingroup$ (III) Even trying to tie the fixed/random effect terminology to fixed/random variable terminology breaks down more generally because in Bayesian models, the fixed effects, even just the predictors in a classical non-mixed model, are technically random variables, as discussed e.g. here. $\endgroup$ – Livius Dec 11 '20 at 14:58
4
$\begingroup$

(Turning my comments into a real answer.)

I think the real question here is "what does 'random effect' refer to"?

There seem to be a few obvious candidates:

  1. the blocking/grouping variable
  2. the varying slopes/intercepts
  3. the associated predictions for the varying slopes/intercepts (i.e. the BLUPs or conditional modes)
  4. the entirety of blocking variable + varying slopes/intercepts (although there is some ambiguity even in this, whether the $Z$ model matrix or the latent random variable $B$ or the observed levels of that variable $b$ are meant in the usual formulation. This ambiguity actually parallels the ambiguity in 1-3, and highlights why these different interpretations actually all make sense because they're getting at different aspects of the same part of the math.)

It turns out that 'random effect' can refer to any and all of these things, depending on context. For example, both lme4 and MixedModels.jl use ranef() to extract the BLUPs, but also refer to the estimated variances for the varying slopes/intercepts as random effects. This ambiguity is discussed at length in many places, e.g. in Gelman and Hill's introductory text (p. 245 from the book published in 2006; there is or soon will be a two-part new 'edition'.)

SIDEBAR: Why the use of "random" cannot be the decisive in resolving this ambiguity I want to make one important note here: the OP is arguing for a terminological Final Verdict based on the term 'random'. First, I don't think we can convince a broad community of users to adopt our preferred terminology by fiat. Second, in the Bayesian formulation of regression, even the fixed effects are random variables, see e.g. Jarrod Hadfield's excellent course notes, which are a vignette in the great MCMCglmm package. The terminological ambiguity is also discussed somewhat there. (This is part of why frequentists have intervals and Bayesians have posterior distributions -- parameters are fixed values in frequentism, whose sampling distribution is a random variable, but the parameters themselves are random variables in Bayesianism.) All of this is horribly simplified to make it fit into a sidebar, so please be gentle when correcting any infelicities from compressing complex ideas into a small space for non experts.

This all seems horrible, but if it's any comfort, even the terminology "fixed effect" can be ambiguous in other contexts. For example, in econometrics "fixed effects" means something like "a categorical covariate" (also discussed in Gelman and Hill, but perhaps most obvious from the name of a Julia package for regression with high dimensional categorical predictors. This has lead some Andrew Gelman to prefer the "multilevel" or "hierarchical" to "mixed-effects", but even these terms are not perfect. For example, one of the innovations in software like lme4 is that you don't have to have strict hierarchical nesting of the blocking variables, so it seems weird to call those models "hierarchical" or even "multilevel" when there isn't necessarily a clear stratification into success "levels". (This is also why I find the Level 1 and Level 2 terminology in some classical texts so confusing.) Additionally, "level" is often used to refer to the different possible realizations of a categorical variable, as in the levels() function in R when applied to factor() variables. Moreover, "hierarchical regression" can refer to a particular procedure for variable selection classical OLS regression (see e.g. this article discussing it in the list of misused terms).

In other words, even if we could declare by fiat that "random effects" shall henceforth refer to only one of the possibilities above, we still have a pile of terminological ambiguities to deal with. This is just part of working a field (statistics) that both has its own set of diverse research traditions and is deeply connected via its users to an even more diverse set of research traditions across many different fields.

How to deal with this then? My usual approach has a few components:

  1. to make it obvious from context (e.g. talking about repeated measures and where the repetition occured)

  2. use alternative terminology to make it clear, e.g. "by-location intercepts" or "varying slopes by location". This is Andrew Gelman's solution. Similarly, Doug Bates will often be explicit about "experimental" and "blocking" variables, e.g. here, and stating which experimental factors are allowed to vary between or within which blocking factors. This "between" and "within" terminology also hints at the deep underlying connection to classical repeated-measures ANOVA.

  3. Use the Wilkinson-Roger formula notation to be very explicit, without being painfully mathematical (which is why I often write out the formulae).

  4. Expanding on (3): include code in research publications! Words are often ambiguous; explicit math and code less so. Mathematical notation was developed because it is generally more precise than words, even if it takes some practice to master this new method of expression.

EDIT 2020-12-11 21:23 UTC The question has now been edited to include the qualifier "frequentist". I don't think this actually helps that much / invalidates my sidebar. Frequentist mixed-effects models are still vaguely Bayesian, which is why "conditional mode" is a more accurate name than BLUPs, and indeed a common frequentist derivation of mixed models is essentially as an empirical Bayes problem. (This is also evident in the derivation of the profiled log likelihood used in lme4 and MixedModels.jl: there is literally a line where you have something that looks mathematically like a prior times a likelihood.) I also think it discards the common ground between the "strict" frequentist / maximum-likelihood approach and the Bayesian approach. Both MCMCglmm and brms use the fixed- and random-effect terminology in their documentation, partly because this is a standard way to emphasize the difference between estimating means and estimating variances and that means for the model parameterization and thus the interpretation of model parameters. Because that's really the difference we're getting at in fixed vs. random effect, with the different meanings of "random effect" largely emphasizing different parts of the structure of that multivariate covariance. That, however, is a discussion that goes beyond the scope of this question and which been touched upon numerous questions on this site, in the documentation to all the software I mentioned and in many introductory texts on this material.

$\endgroup$
0
$\begingroup$

In the frequentist setting, parameters are either be fixed, unknown quantities, or modeled as random variables. They aren't both.

However, I know we actually take the intercepts or slopes (or both) of a fixed-effect variable (e.g., A) to vary (i.e., be random) among each level of a grouping variable (X).

Slopes being different across a subgroup variable doesn't mean they're random. You could have a model with 50 fixed, unknown slope parameters.

Question: So, shouldn't the the discussion regarding whether to take a variable both as fixed and random effect or not be represented for instance by the following syntax

I didn't understand your syntax idea, but again, a variable "both as a fixed and random effect" doesn't make sense to me.

Perhaps you mean something like:

$$y_{ij} = \alpha_i + \beta X_{ij} + \epsilon_{ij}$$ where $$\alpha_i \sim N(\bar{\alpha}, \sigma_{\alpha}^2)$$

which can be rewritten as

$$y_{ij} = (\bar{\alpha} + a_i) + \beta X_{ij} + \epsilon_{ij}$$ where $$a_i \sim N(0, \sigma_{\alpha}^2)$$ so that the intercept has both a fixed and random effect component.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.