2
$\begingroup$

I have read through the emmeans "Basics of EMMs" vignette, "Working with messy data" vignette, and this Stack Overflow post but am still having a hard time knowing which 'weights' argument to use when factor levels have uneven numbers of observations.

I am using binomial GAMs with mgcv in R to model binary species presence on a hydrophone against environmental (chlorophyll-a concentration, sea-surface temperature, sea-level, vessel presence) and categorical temporal covariates (season and photoperiod):

M <- gam(Species ~ s(Chla,bs="ts") + s(SST,bs="ts") + s(SLEV,bs="ts") + Season + Photoperiod + Vessel, data=Hydrophone, family='binomial', method = "ML")

The categorical groups do not have an even number of observations (i.e. dawn/dusk photoperiods are shorter than night/day, fewer observations were gathered in summer than other season, clips with vessel presence/absence are uneven) and I am hoping to account for this when running pairwise comparisons in contrast from the emmeans library on my model factors. Specifically, I don't want small groups to have less weight than large groups; they should be proportional to the total number of observations per group.

I am primarily torn between using weights="proportional" and weights="cells". The description for weights="proportional" in the emmeans help file makes it sound ideal since it uses "Weight in proportion to the frequencies (in the original data) of the factor combinations that are averaged over." However, when I run weights="proportional" in contrasts I get the same results as when I run weights="equal", which doesn't seem right.

weights="cells" uses "Weight according to the frequencies of the cells being averaged" but this is in regards to the reference grid constructed by emmeans and I am unsure if those frequencies are the same as in the original dataset? Using weights="cells" in contrast gives different results than weights="proportional" and weights="equal".

To summarize, my question is which weighting argument accounts for unbalanced factors (i.e. different number of observations per factor level) when conducting pairwise comparisons using contrast from the emmeans R library?

Sorry for the long post but I wanted to provide adequate context.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

The issue is not accounting for unbalanced data, but what kind of inference you want to make. Are you trying to characterize a population from which you sampled, or did you vary those factor levels experimentally? The extremes are "equal" (the default), which is best for estimating the effects of one factor while holding the rest fixed in an experimental mode; and "cells", which in a linear model, basically reproduces the raw marginal means of the data but is least useful for quantifying effects and most vulnerable to confounding with effects of other factors. You might want to also consider counterfactuals (look that up in the vignette index) which allow for certain kinds of causal inferences.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks for the information. This study is definitely observational, and I know default emmeans leans towards the experimental designs. Based on your answer it seems that "cells" might be the best option since there were no experimental treatments/controls being applied. I am not looking to perfectly isolate the effects of individual factors, just characterizing patterns in the population, so I think I am willing to accept some confounding effects. $\endgroup$
    – Roanan
    Mar 28 at 17:04
  • $\begingroup$ Still, you might consider weights = "outer", which is proportional weighting iterated over one variable averaged over at a time. It is equivalent to making the cell weights equal to the expected frequencies in a chi-square test of independence; so it makes the cell weights independent of the marginal frequencies. $\endgroup$
    – Russ Lenth
    Mar 28 at 23:23
  • $\begingroup$ This might have something to do with my model specification but I am getting the same pairs() results whether I run emmeans() with "outer", "proportional", "equal", or "flat" weights? The only weights specification producing different results for pairs() is weights="cells". $\endgroup$
    – Roanan
    Mar 29 at 2:50
  • $\begingroup$ If there is only one variable to average over, outer and proportional are the same. If the total weights for each level of a variable are the same, then proportional weighting will be the same as equal weighting. You can see what weights are used by doing ref_grid(model)@grid $\endgroup$
    – Russ Lenth
    Mar 29 at 17:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.