# Comparing differences between overlapping groups

I am trying to analyse the differences between people who vote in the European Parliament (EP) elections and people who vote in national elections, based on the EES dataset$$^\dagger$$. The objective is to find significant differences between these two groups on socio-demographical variables (age, gender, etc.) The problem is that these groups are partially overlapping (some people vote in both elections) and the predictors are mostly categorical (ordinal). What statistical techniques and error estimates can I use to tackle this? (Is factorial ANOVA okay?) Is there a way to combine error estimates for repeated measures and independent cases?

$$^\dagger$$ updated site link.

With overlapping groups, you can still use a t-test based on the following:

\begin{aligned} \text{Under }H_0:\space \bar{Y}_1 = \bar{Y}_2 &\\ \\ \frac{\hat{\bar{Y}}_1 - \hat{\bar{Y}}_2}{\sqrt{\operatorname{var}(\hat{\bar{Y}}_1 - \hat{\bar{Y}}_2)}} &\sim t(df) \end{aligned}

The tricky part is that in order to estimate the variance term in this test statistic, you have to make sure to take into account the correlation between $$\hat{\bar{Y}}_1$$ and $$\hat{\bar{Y}}_2$$ caused by the overlap. Also, it's not clear what you should use for the degrees of freedom, $$df$$; you could probably do well to just use the rule-of-thumb design degrees of freedom $$(\#\text{ of clusters} - \#\text{ of strata})$$.

## Option #1: Use replicate weights

You can do this easily using replication methods (jackknife, bootstrap, etc.) if you have sets of replicate weights: simply use each set of replicate weights to calculate the two estimates and their difference, and then calculate the variance of the estimated difference across the sets of replicate weights, using the appropriate method for how your replicate weights were formed. If you're working with the survey R package, you can use the withReplicates() function to do this.

library(survey)
library(srvyr)

# Load survey of schools ----
data(api, package = "survey")

# Create survey design object with replicate weights ----
dstrata <- apistrat %>%
as_survey(strata = stype, weights = pw) %>%
# Generate two overlapping groups
mutate(meals_0_to_60 = ifelse(meals >= 0 & meals <= 60,
1, 0),
meals_50_to_100 = ifelse(meals >= 50 & meals <= 100,
1, 0))

dstrata_rep <- as_survey_rep(dstrata)

# Use the withReplicates() function to estimate standard error of differnces ----
diff_in_means <- function(svy_weights, svy_vbls) {
ybar_1_num = sum(svy_weights * svy_vbls[['meals_50_to_100']] * svy_vbls[['api00']])
ybar_1_dem = sum(svy_weights * svy_vbls[['meals_50_to_100']])
ybar_1 <- ybar_1_num / ybar_1_dem

ybar_2_num = sum(svy_weights * svy_vbls[['meals_0_to_60']] * svy_vbls[['api00']])
ybar_2_dem = sum(svy_weights * svy_vbls[['meals_0_to_60']])
ybar_2 <- ybar_2_num / ybar_2_dem

ybar_1 - ybar_2
}

withReplicates(dstrata_rep, theta = diff_in_means)
#>        theta    SE
#> [1,] -159.18 13.11


## Option #2: Use the method of linearization by influence functions

You can also use linearization methods to get this variance estimate. Check the blog post explaining how to do this with the survey R package and why it works.

An example with the school survey dataset from the example code is as follows:

library(survey)
library(srvyr)

# Load survey of schools ----
data(api, package = "survey")

# Create survey design object ----
dstrata <- apistrat %>%
as_survey(strata = stype, weights = pw) %>%
# Generate two overlapping groups
mutate(meals_0_to_60 = ifelse(meals >= 0 & meals <= 60,
1, 0),
meals_50_to_100 = ifelse(meals >= 50 & meals <= 100,
1, 0))

# Calculate means for the two groups ----

ybar_1 <- svyratio(numerator = ~ I(meals_50_to_100 * api00),
denominator = ~ meals_50_to_100,
design = dstrata, influence = TRUE)

ybar_2 <- svyratio(numerator = ~ I(meals_0_to_60 * api00),
denominator = ~ meals_0_to_60,
design = dstrata, influence = TRUE)

diff_in_means <- ybar_1$$ratio - ybar_2$$ratio

# Obtain influence functions of the two statistics ----

dstrata$$variables[['ybar_1_infl']] <- attr(ybar_1, 'influence') * dstrata$$prob
dstrata$$variables[['ybar_2_infl']] <- attr(ybar_2, 'influence') * dstrata$$prob

# Use the influence functions to estimate variances and covariance ----

infl_totals <- svytotal(x = ~ ybar_1_infl + ybar_2_infl,
design = dstrata)

vcov_of_ybar_estimates <- vcov(infl_totals)

# Estimate standard error of difference ----

var_of_diff <- t(c(1,-1)) %*% vcov_of_ybar_estimates %*% c(1,-1)
std_err_of_diff <- sqrt(var_of_diff)

#>          [,1]
#> [1,] 13.02953


What I would do is make a new dependent variable with 4 levels: Both, EU only, national only, and neither and use "neither" as the reference category. Then do multinomial logistic regression.

As for ordinal IVs, those are tricky, but see this thread for lots of discussion and ideas.