7
$\begingroup$

I am trying to understand the meaning of the effective sample size when we have weights for the sample. I have an intuitive understanding of why effective sample size can be less than the raw sample size when correlations exist in the data, but can somebody please explain why this is the case when the sample is weighted? Can weights be regarded as introducing some kind of correlation into the data? Thanks!

$\endgroup$
5
  • 1
    $\begingroup$ An illustration: if I have five samples, and weight one of them at zero, I effectively have four samples. $\endgroup$ Oct 16, 2018 at 15:14
  • $\begingroup$ That's an extreme example. Also, when we weight a sample, we usually make sure that the sum of the targets for different exclusive cuts add up to 1. In your example, if we weight one respondent to 0, the other 4 should be weighted up by a factor of 1.25. $\endgroup$ Oct 17, 2018 at 18:02
  • 1
    $\begingroup$ Our weights might be for different purpose, as I use regression weights to represent the statistical uncertainty in a sample's value where the greater the uncertainty the smaller the weight. My example was meant to illustrate effective sample size and not as a discussion of statistical uncertainty in relation to weighted regression. $\endgroup$ Oct 17, 2018 at 18:13
  • $\begingroup$ What sort of weights are you referring to? Design weights? $\endgroup$
    – num_39
    Aug 22, 2022 at 16:04
  • $\begingroup$ I just mean weights that are used to adjust the representations of different sub-groups of the sample (like males and females). $\endgroup$ Aug 31, 2022 at 14:12

1 Answer 1

5
+100
$\begingroup$

Can weights be regarded as introducing some kind of correlation into the data?

Assume you have two uncorrelated standard normal variables with lots of samples. If all points on / closest to a positively sloped line are heavily weighted relative to other points, you would induce positive correlation. Similarly, if all points on / closest to a negatively sloped line are heavily weighted relative to other points, you would induce negative correlation.

Sometimes weighting points near a line is intentional, see for instance RANSAC (random sample consensus).

I am trying to understand the meaning of the effective sample size when we have weights for the sample.

In a sample with independent identically distributed (i.i.d.) observations, precision (inverse of variance) of parameter estimates (e.g. mean $\hat{\mu}$) is additive, accumulating over the sample observations. If you weight your sample observations or your sample is not i.i.d, the precision of your parameter estimate and the effective size of your sample is reduced relative to an equally weighted i.i.d. sample with the same number of observations.

Intuitively effective sample size is your actual sample size scaled by the ratio of precision of a parameter estimate using your weighted sample divided by the precision of the same parameter estimate in a counterfactual i.i.d. sample.

For purposes of intuitively understanding reduced effective sample size, consider

$$N_\textrm{effective} \approx N_\textrm{actual} \frac{\textrm{Precision}_W(\hat{\mu})}{\textrm{Precision}_\textrm{iid}(\hat{\mu})} = N_\textrm{actual} \frac{\textrm{Var}_\textrm{iid}(\hat{\mu})}{\textrm{Var}_W(\hat{\mu})} $$

See discussion and references in Effective sample size for more details.

$\endgroup$
2
  • 2
    $\begingroup$ "the ratio of precision of a parameter estimate using your weighted sample divided by the precision of the same parameter estimate in a counterfactual i.i.d. sample" This is a very good characterisation. It also shows how the question is though to answer: when the true sample has different weights then there is no clear way to come up with a counterfactual i.i.d. sample. In the case of correlations you can remove the correlations, but in the case of different variance of the measurements in the sample, which variance are you gonna choose? $\endgroup$ Aug 25, 2022 at 6:19
  • $\begingroup$ Is there a way you could explicitly show how weighting leads to less precision? For example, if I have $X_1, \ldots, X_n$, how would it look like weighted? $\endgroup$
    – user321627
    Aug 27, 2022 at 6:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.