I am currently working on a random intercept multilevel model using the European Social Survey round 6 dataset. It is a 2-level model with individuals (level 1) nested within countries (level 2). To simplify things, imagine the following regression:
$Y_{ij}=\beta_{0j}+\beta_{1j}X_{ij}+e_{ij}$
where the dependent variable is trust in the European Parliament on a scale from 0-10, and the level-1 predictor is gender. In the data set, there exists two weights:
- Design weight: The design weights are inclusion probabilities for individuals $i$ in countries $j$. The design weight corrects for slightly different probabilities of selection, thereby making the sample more representative of a ‘true’ sample of individuals from each country.
- Population size weights: The population size weight makes an adjustment to ensure that each country is represented in proportion to its population size. The population size weight is calculated as PWEIGHT= [Population size]/[(Net sample size in data file)*10 000]
My question is: do I need to specify the population size weights when I run the multilevel model? I tend to get different results. Below is the regression with design weights apllied (I am using Stata):
. xtmixed trstep gndr [pw = dweight]|| land:, mle var
Obtaining starting values by EM:
Performing gradient-based optimization:
Iteration 0: log pseudolikelihood = -92442,22
Iteration 1: log pseudolikelihood = -92442,22 (backed up)
Computing standard errors:
Mixed-effects regression Number of obs = 39923
Group variable: land Number of groups = 24
Obs per group: min = 579
avg = 1663,5
max = 2711
Wald chi2(1) = 5,91
Log pseudolikelihood = -92442,22 Prob > chi2 = 0,0151
(Std. Err. adjusted for 24 clusters in land)
------------------------------------------------------------------------------
| Robust
trstep | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
gndr | ,1147821 ,0472334 2,43 0,015 ,0222063 ,2073578
_cons | 4,144926 ,117911 35,15 0,000 3,913825 4,376027
------------------------------------------------------------------------------
------------------------------------------------------------------------------
| Robust
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
land: Identity |
var(_cons) | ,3184852 ,0689119 ,2084065 ,4867066
-----------------------------+------------------------------------------------
var(Residual) | 5,93535 ,2514202 5,462477 6,449158
------------------------------------------------------------------------------
And here is the regression with both design weights, population size weights and scaling applied:
. xtmixed trstep gndr [pw = dweight]|| land:, mle var pweight(pweight) pwscale(size)
Obtaining starting values by EM:
Performing gradient-based optimization:
Iteration 0: log pseudolikelihood = -81334,099
Iteration 1: log pseudolikelihood = -81333,24
Iteration 2: log pseudolikelihood = -81333,24
Computing standard errors:
Mixed-effects regression Number of obs = 39923
Group variable: land Number of groups = 24
Obs per group: min = 579
avg = 1663,5
max = 2711
Wald chi2(1) = 10,73
Log pseudolikelihood = -81333,24 Prob > chi2 = 0,0011
(Std. Err. adjusted for 24 clusters in land)
------------------------------------------------------------------------------
| Robust
trstep | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
gndr | ,1680609 ,0513105 3,28 0,001 ,0674942 ,2686276
_cons | 3,745146 ,1854299 20,20 0,000 3,38171 4,108582
------------------------------------------------------------------------------
------------------------------------------------------------------------------
| Robust
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
land: Identity |
var(_cons) | ,2645594 ,0598105 ,1698583 ,412059
-----------------------------+------------------------------------------------
var(Residual) | 6,070198 ,3532338 5,415894 6,803549
------------------------------------------------------------------------------
I can't figure out how the population weights influence the ML estimates. The official ESS documentation says:
When comparing data from two or more countries but without reference to the average (or combined total) of those countries, only the design weight need be applied. When comparing data of two or more countries and with reference to the average (or combined total) of those countries, both design and population size weights should be applied. (ESS Documentation)
The question is, whether I actually just compare countries, or compare them to an overall mean. The latter seems to be correct, as the parameter estimates actually relate to the overall mean $\mu_{00}$. But I may be wrong. The Stata manual has an extensive section on weighting here but it did not clear things up for me. Any suggestions?
