When to use fixed effects or multi level models in regression?

Question

Suppose you run an experiment where the treatment is Gatorade and the outcome is one-mile runtime. You’ve stratified on variables such as sex, height and weight so they’re well randomized and have no correlation with the treatment. Terrific!

Now suppose that you’re interested in runtime and sex (using a categorical variable for simplicity). Because you stratified on sex, you can condition on it in your post-experiment inference. To this end, I see two options.

1. Include a coefficient for Gatorade and a coefficient for sex (ex: variable is $1$ for female else $0$.)

2. Use a multilevel model where there’s a global slope and sub-group slopes (one for man and one for female). The global slope is a prior for each of the sub group slopes.

My questions are: Are these methods equivalent? If not, when would one be more appropriate than the other?

Answer 1

Theoretically, there is nothing stopping you from using a two-level random effect, but I think a mixed model here would be complete overkill. I would just fit a model that treats sex as a categorical variable and leave it at that. With a fixed effects version, you get directly interpretable coefficients. A mixed model would be more useful if you had several clusters that you were trying to summarize in a meaningful way, but here you just have two. You don't know yet if they meaningfully vary enough to use as random effects (which could be its own problem with convergence if they don't). So it would be best to just stick to the fixed effects version in my opinion.

As far as whether or not you can use Bayes, there's nothing stopping you from using Bayes for fixed or random effects. That is more on you for deciding which paradigm you want to utilize, one with priors and one without.

Answer 2

Like Shawn, I see no need for a mixed model (aka multilevel model). There's no reason your errors should not be independent in the "regular" regression and that is the problem that MLM or mixed models are designed to solve.

But I would include an interaction between sex and Gatorade. It seems to me that what is interesting here is whether Gatorade has different effects on the runtime of men and women, and that is what an interaction tests.