How to calculate the standard error of a proportion using weighted data?

I know the "textbook" estimate of the standard error of a proportion is $SE=\sqrt{\frac{p(1-p)}{n}}$, but does this hold up when the data are weighted?

• This is the root of the inverse of the Fisher information for a binomial distribution. The Fisher information is the variance of the expected value of the observed information. It is the standard deviation of the expected error. This expression should be valid for all binomial distributions. In practice, if the probability is quite close to one or to zero while you have few samples, the value given by the expression might have large error. Make sure your sample sizes are large enough. – EngrStudent Jun 29 '15 at 17:59

Yes, this formula generalizes in a natural way.

Standardize the (positive) weights $\omega_i$ so they sum to unity.

In a simple random sample $X_1, \ldots, X_n$ where each $X_i$ independently has a Bernoulli$(p)$ distribution and weight $\omega_i$, the weighted sample proportion is

$$\bar X = \sum_{i=1}^n \omega_i X_i.$$

Since the $X_i$ are independent and each one has variance $\text{Var}(X_i) = p(1-p)$, the sampling variance of the proportion therefore is

$$\text{Var}(\bar X) = \sum_{i=1}^n \text{Var}(\omega_i X_i) = p(1-p)\sum_{i=1}^n\omega_i^2.$$

The standard error of $\bar X$ is the square root of this quantity. Because we do not know $p(1-p)$, we have to estimate it. Although there are many possible estimators, a conventional one is to use $\hat p = \bar X$, the sample mean, and plug this into the formula. That gives

$$\text{SE}(\bar X) = \sqrt{\bar X(1-\bar X) \sum_{i=1}^n \omega_i^2}.$$

For unweighted data, $\omega_i = 1/n$, giving $\sum_{i=1}^n \omega_i^2 = 1/n$. The SE becomes $\sqrt{p(1-p)/n}$ and its estimate from the sample is $\sqrt{\bar X(1-\bar X)/n}$. These are the familiar formulas, showing that the calculation for weighted data is a direct generalization of them.

It depends what you mean by `weighted'.

If you mean sampling weights from a survey design there isn't a simple extension of the $p(1-p)/n$ rule except under some quite strong additional assumptions (independent sampling of individuals, with weight independent of the value of $X$).

For example, suppose you had stratified sampling stratified on the value of $X$. The variance of $\bar X$ would be zero. More realistically, you might have stratified sampling on a variable very close to $X$ -- $X$ measured with error, in some sense -- and the variance would be small.

In the other direction, if you had cluster sampling, or independent sampling uncorrelated with the value of $X$, you'd get a higher variance than the $p(1-p)/n$ rule would give.