3
$\begingroup$

I have a large data set of 20M observations with 20+ measures for each observation (people). Additionally I have a set of weights that reflect the racial population where each person lives. For example, from Census data this could be the percentage of population in the zipcode, or tract that is African American, White, Asian etc. These percentages will add to 100%.

We lack direct measure of race on individuals. This information is not available.

The profile data will then look like this:

ID,White,Asian,AfricanAmerican,...
1234    70% 20% 5%  5%  0%
2345    30% 50% 10% 5%  5%
3456    5%  90% 5%  0%  0%
4567    25% 25% 25% 15% 10%

The 20 measures are a combination of financial data and true/false flags. These finance data are heavily skewed, and also have a lot of zeroes.

We will compute the weighted means of all the measures, e.g. average value of African Americans.

My questions center around computing variance measures for these means.

I think the most straightforward way to compute variance estimates is to bootstrap the IDs, preserving the profile of each ID and compute $N$ weighted means.

Alternatively, we could randomly pick one race per person $N$ times according to the profile and compute $N$ weighted means. We could also do both, re-sample the IDs and pick 1 race in each iteration.

I cant figure out what the "one race" simulation results actually measure. Because each person is assigned one race, it can't be used for a confidence interval of the original weighted means.

Is there a name for this second method of simulating the random variable? Any references for using it?

$\endgroup$
1
  • $\begingroup$ What are you estimating when you weight individuals' scores with weights derived at a group level? Are you intending to estimate eg average white and average asian scores for each variable? Because this may not be an effective way of doing it. Ecological fallacy etc. Do you have racial information on individuals? $\endgroup$ Jul 19, 2013 at 23:11

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.