Understanding the Weighted Bootstrap

Question

I have recently learned about the idea that it is possible to bootstrap "weighted" data - for example, suppose we have:

• A set of sample means : $\bar{x}_1,\bar{x}_2,\ldots,\bar{x}_k$
• The sample size used to calculated each sample mean: $n_1, n_2\ldots,n_k$
• The population size from which each sample mean was taken: $N_1, N_2,\ldots,N_k$
• The sample variance of each mean: $\operatorname{Var}(\bar{x}_1), \operatorname{Var}(\bar x_2),\ldots,\operatorname{Var}(\bar x_k)$

Let's further assume that we decide to assign a "weight" $w_i$ to each of these sample means (e.g. based on counts, based on variance, etc.) - how do we now bootstrap this weighted data?

The first thing that comes to mind is the following:

• In the regular bootstrap method, we repeatedly take random samples with replacement from the original data, calculate the mean of each random sample - and then create a histogram of all the samples.
• Is it possible that in the weighted bootstrap method - we repeat this same process, but now the probability of selecting any observation is proportional to its weight?

Below, I have tried to write the R code as to how I believe weighted data would be bootstrapped:

# function to calculate the weighted mean (inputs: data x and weights w)

weighted_mean <- function(x, w) {
  sum(x * w) / sum(w)
}

# function that performs random sampling with replacement where the probability of selecting any point is proportional to the assigned weight (inputs: R is the number of bootstrap repetitions) 


weighted_bootstrap <- function(data, weights, R) {   
  estimates <- numeric(R)  
  for (i in seq_len(R)) {
    bootstrap_sample <- sample(data, size = length(data), replace = TRUE, prob = weights)
    estimates[i] <- weighted_mean(bootstrap_sample, weights)
  }
  estimates
}

Here is how this weighted bootstrap function would be used on some data (note that the weights must add to 1) :

data <- c(1, 2, 3, 4, 5)
weights <- c(0.1, 0.2, 0.3, 0.2, 0.2)
R <- 1000
estimates <- weighted_bootstrap(data, weights, R)
plot(hist(estimates))

My Question: Can someone please tell me if I have implemented this correctly - can the weighted bootstrap method really be implemented as such (and use weights proportional to the counts or variance)?

Answer 1

The code related to your problem should be a little bit modified as follows:

  estimates <- numeric(R)  
  for (i in seq_len(R)) {
    bootstrap_sample <- sample(data, size = length(data), replace = TRUE, prob = weights)
    estimates[i] <- mean(bootstrap_sample) #HERE IS THE CHANGE
  }
  estimates
}


data <- c(1, 2, 3, 4, 5)
weights <- c(0.1, 0.2, 0.3, 0.2, 0.2)
R <- 1000
estimates <- weighted_bootstrap(data, weights, R)
plot(hist(estimates))

So you don't double-count weights, and you don't assign the wrong weight to a given component: in other words, you don't assign the weight $w_i$ to the sample $x_j$ for $j \neq i$. You'll get the following histogram:

Now, as you can see and as we might expect, the distribution of averages is slightly skewed to the right. In your previous graph, the distribution is perfectly symmetrical, as if the weighted mean distribution converged to a normal distribution, which does not conform to the "classical" central limit theorem, see this topic.