Given the Bayesian paradigm, is there actually such a thing as a fixed effects model since Bayesians treat all parameters as random variables?


An excerpt from Jospeh B. Kadane's Principles of Uncertainty (p. 336, freely available):

This idea has old historical roots. Because these roots still play out in the current literature, it is useful to retrace a bit of them. The received wisdom in the early 1960's (see for example Scheffe (1959, 1999)) was to draw a distinction in linear models between “fixed effects” and “random effects”. “Mixed effect models" had both random effects and fixed effects. And what was the diference between random effects and fixed effects? It had to do with what you were interested in. If you were interested in the ability of each child, you would treat the $\alpha$’s as fixed effects parameters. If you were interested in the classes, but not the ability of each child, you would treat the $\beta$’s as random effects and the $\alpha$’s as fixed effects in the example. There are several peculiarities in this from a Bayesian point of view. First, “random effects” are parameters with priors. The classical analysis integrates those parameters out of the likelihood. But classically parameters are not supposed to have distributions, and integrating with respect to a parameter is supposedly an illegitimate move. Second, the distinction between “random" and “fixed” is essentially about what one wishes to estimate, and thus is a matter of the utility function. How can it be that the utility function can affect what the likelihood is, particularly in a classical context in which the likelihood is imagined to be the objective truth about how the data were generated? Third, what if I care both about the children individually and about how classes of children compare? I can't treat the same parameter as both fixed and random in the same analysis! I can remember confused social scientists wanting advice about which parameters to treat as random and which as fixed, and being surprised at the response that all parameters are random (i.e., are uncertain quantities that have distributions). From a Bayesian perspective there is no distinction, and no issue.

  • 5
    $\begingroup$ This is a great answer. I would further complicate the issue by referencing Gelman and Hill's attack against the "fixed" and "random" terminology itself in their book on hierarchical modeling, where they outline the four (!) or so distinct usages of the terms "fixed effect" and "random effect" which the authors have discovered in the literature. $\endgroup$
    – Sycorax
    Aug 25 '14 at 21:05

Not the answer you're looking for? Browse other questions tagged or ask your own question.