I am currently working on a random intercept multilevel model using the European Social Survey round 6 dataset. It is a 2-level model with individuals (level 1) nested within countries (level 2). To simplify things, let's look at the following regression:
$Y_{ij}=\beta_{0j}+\beta_{1j}X_{ij}+e_{ij}$
where the dependent variable is 'trust in the European Parliament' on a scale from 0-10, and the level-1 predictor is gender. In the data set, two weights exists:
- Design weight: The design weights are inclusion probabilities for individuals $i$ in countries $j$. The design weight corrects for slightly different probabilities of selection, thereby making the sample more representative of a ‘true’ sample of individuals from each country.
- Population size weights: The population size weight makes an adjustment to ensure that each country is represented in proportion to its population size. The population size weight is calculated as PWEIGHT= [Population size]/[(Net sample size in data file)*10 000]
My question is now - do I need to specify the population size weights when I run the multilevel model? I clearly get different results. Below is the regression only using design weights (I am using STATA):
. xtmixed trstep gndr [pw = dweight]|| land:, mle var
Obtaining starting values by EM:
Performing gradient-based optimization:
Iteration 0: log pseudolikelihood = -92442,22
Iteration 1: log pseudolikelihood = -92442,22 (backed up)
Computing standard errors:
Mixed-effects regression Number of obs = 39923
Group variable: land Number of groups = 24
Obs per group: min = 579
avg = 1663,5
max = 2711
Wald chi2(1) = 5,91
Log pseudolikelihood = -92442,22 Prob > chi2 = 0,0151
(Std. Err. adjusted for 24 clusters in land)
------------------------------------------------------------------------------
| Robust
trstep | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
gndr | ,1147821 ,0472334 2,43 0,015 ,0222063 ,2073578
_cons | 4,144926 ,117911 35,15 0,000 3,913825 4,376027
------------------------------------------------------------------------------
------------------------------------------------------------------------------
| Robust
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
land: Identity |
var(_cons) | ,3184852 ,0689119 ,2084065 ,4867066
-----------------------------+------------------------------------------------
var(Residual) | 5,93535 ,2514202 5,462477 6,449158
------------------------------------------------------------------------------
And here is the regression using both deisgn weights, population size weights and scaling:
. xtmixed trstep gndr [pw = dweight]|| land:, mle var pweight(pweight) pwscale(size)
Obtaining starting values by EM:
Performing gradient-based optimization:
Iteration 0: log pseudolikelihood = -81334,099
Iteration 1: log pseudolikelihood = -81333,24
Iteration 2: log pseudolikelihood = -81333,24
Computing standard errors:
Mixed-effects regression Number of obs = 39923
Group variable: land Number of groups = 24
Obs per group: min = 579
avg = 1663,5
max = 2711
Wald chi2(1) = 10,73
Log pseudolikelihood = -81333,24 Prob > chi2 = 0,0011
(Std. Err. adjusted for 24 clusters in land)
------------------------------------------------------------------------------
| Robust
trstep | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
gndr | ,1680609 ,0513105 3,28 0,001 ,0674942 ,2686276
_cons | 3,745146 ,1854299 20,20 0,000 3,38171 4,108582
------------------------------------------------------------------------------
------------------------------------------------------------------------------
| Robust
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
land: Identity |
var(_cons) | ,2645594 ,0598105 ,1698583 ,412059
-----------------------------+------------------------------------------------
var(Residual) | 6,070198 ,3532338 5,415894 6,803549
------------------------------------------------------------------------------
I can't figure out how the population weights influence the ML estimates. In the official documentation for ESS, they write the following:
When comparing data from two or more countries but without reference to the average (or combined total) of those countries, only the design weight need be applied. When comparing data of two or more countries and with reference to the average (or combined total) of those countries, both design and population size weights should be applied. (ESS Documentation)
The question is, whether I actually just compare countries, or compare them to an overall mean. The latter seems for me to be correct, as the parameter estimates actually relate to the overall mean $\mu_{00}$. But I may be wrong. The STATA manual has an extensive section on weighting here but it did not clear things up for me. Any suggestions?