I am currently working on a random intercept multilevel model using the European Social Survey round 6 dataset. It is a 2-level model with individuals (level 1) nested within countries (level 2). To simplify things, let's look at the following regression:
$Y_{ij}=\beta_{0j}+\beta_{1j}X_{ij}+e_{ij}$
where the dependent variable is **trust in the European Parliament** on a scale from 0-10, and the level-1 predictor is **gender**. In the data set, two weights exists:
- _Design weight_: The design weights are inclusion probabilities for individuals $i$ in countries $j$. The design weight corrects for slightly different probabilities of selection, thereby making the sample more representative of a ‘true’ sample of individuals from each country.
- _Population size weights_: The population size weight makes an adjustment to ensure that each country is represented in proportion to its population size. The population size weight is calculated as PWEIGHT= [Population size]/[(Net sample size in data file)*10 000]
My question is: do I need to specify the population size weights when I run the multilevel model? I clearly get different results. Below is the regression only using design weights (I am using Stata):
. xtmixed trstep gndr [pw = dweight]|| land:, mle var
Obtaining starting values by EM:
Performing gradient-based optimization:
Iteration 0: log pseudolikelihood = -92442,22
Iteration 1: log pseudolikelihood = -92442,22 (backed up)
Computing standard errors:
Mixed-effects regression Number of obs = 39923
Group variable: land Number of groups = 24
Obs per group: min = 579
avg = 1663,5
max = 2711
Wald chi2(1) = 5,91
Log pseudolikelihood = -92442,22 Prob > chi2 = 0,0151
(Std. Err. adjusted for 24 clusters in land)
------------------------------------------------------------------------------
| Robust
trstep | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
gndr | ,1147821 ,0472334 2,43 0,015 ,0222063 ,2073578
_cons | 4,144926 ,117911 35,15 0,000 3,913825 4,376027
------------------------------------------------------------------------------
------------------------------------------------------------------------------
| Robust
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
land: Identity |
var(_cons) | ,3184852 ,0689119 ,2084065 ,4867066
-----------------------------+------------------------------------------------
var(Residual) | 5,93535 ,2514202 5,462477 6,449158
------------------------------------------------------------------------------
And here is the regression using both design weights, population size weights and scaling:
. xtmixed trstep gndr [pw = dweight]|| land:, mle var pweight(pweight) pwscale(size)
Obtaining starting values by EM:
Performing gradient-based optimization:
Iteration 0: log pseudolikelihood = -81334,099
Iteration 1: log pseudolikelihood = -81333,24
Iteration 2: log pseudolikelihood = -81333,24
Computing standard errors:
Mixed-effects regression Number of obs = 39923
Group variable: land Number of groups = 24
Obs per group: min = 579
avg = 1663,5
max = 2711
Wald chi2(1) = 10,73
Log pseudolikelihood = -81333,24 Prob > chi2 = 0,0011
(Std. Err. adjusted for 24 clusters in land)
------------------------------------------------------------------------------
| Robust
trstep | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
gndr | ,1680609 ,0513105 3,28 0,001 ,0674942 ,2686276
_cons | 3,745146 ,1854299 20,20 0,000 3,38171 4,108582
------------------------------------------------------------------------------
------------------------------------------------------------------------------
| Robust
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
land: Identity |
var(_cons) | ,2645594 ,0598105 ,1698583 ,412059
-----------------------------+------------------------------------------------
var(Residual) | 6,070198 ,3532338 5,415894 6,803549
------------------------------------------------------------------------------
I can't figure out how the population weights influence the ML estimates. In the official documentation for ESS, they write the following:
> When comparing data from two or more countries but without reference
> to the average (or combined total) of those countries, only the design
> weight need be applied. When comparing data of two or more countries
> and with reference to the average (or combined total) of those
> countries, both design and population size weights should be applied. ([ESS Documentation][1])
The question is, whether I actually just compare countries, or compare them to an overall mean. The latter seems for me to be correct, as the parameter estimates actually relate to the overall mean $\mu_{00}$. But I may be wrong. The Stata manual has an extensive section on weighting [here][2] but it did not clear things up for me. Any suggestions?
[1]: http://www.europeansocialsurvey.org/docs/methodology/ESS_weighting_data.pdf
[2]: http://www.stata.com/bookstore/stata12/pdf/xt_xtmixed.pdf#page=41