I’m investigating possible correlations between the COVID-19 vaccination rate in the United States and the results of a long-running survey of scientific personality traits like "Agreeableness" and "Conscientiousness." While the survey does not ask about vaccination status, it does (optionally and anonymously) ask for self-reported state of residence, age, gender and/or race/ethnicity. This was all conducted under IRB supervision and limited to those age 18 or over. It is a public survey anyone can take, not a poll, so it is not a representative sample.
My challenge is to estimate each respondent's odds of being fully vaccinated based on his or her location and demographic info, taking the state-level vaccination rate and somehow adjusting it with national data on vaccination status by age, gender and race/ethnicity, all from the CDC.
TL;DR: How can I adjust one probability, based on a smaller population (state of residence) with extra information (national variance by age, gender and race/ethnicity) to make the estimate as precise as possible?
(I suspect this is the same exercise a car insurance company conducts when choosing the cost of a policy, using both traffic and crime data in your locality and the odds that other people in your demographic cohort are more or less likely to get in accidents.)
My instinct is to start with the completed vaccination rate in each respondent's state and then simply add or subtract to that value based on how many percentage points more or less likely that individual's age group, gender and race/ethnicity group is to be vaccinated compared to the total population on a national level. But even if this is the correct instinct, it gets tricky because there are two ways of looking at the predictive power of a demographic characteristic.
For example, if I know you live in Delaware and are age 18 or older, I know from CDC data (updated daily) that there's a 68.5% chance you're fully vaccinated (both doses of Pfizer/Moderna or one of J+J). If you're age 65 or over, the rate jumps to 87.3%. That's the only age breakdown the CDC offers at a state level.
But let's say I also know you're 44 years old. The way the CDC reports demographic data is by simply publishing both the percentage of fully vaccinated people in each age group as well as the Census estimate for the percentage of the total population in that age group. So I know that, while only 12.2% of the population is between ages 40 and 49, 14.2% of fully vaccinated individuals are in this age range. I suspect the reason the CDC reports the data in this way is that age is only available for 91% of those vaccinated.
So how do I adjust your base rate of 68.5% with the fact that, across the entire population, those between 40 and 49 years old are over-represented among the fully vaccinated population? The most obvious answer is to add 2.0 percentage points, the difference between 12.2% and 14.2%.
But this may be foolish, because we're dealing with two different types of measurements. The state-level rate is just the percentage of people in the 18+ bracket who are fully vaccinated in each state, regardless of how many of them there are--thus allowing one to easily compare Wyoming to California. Meanwhile, the more precise age-level data for the nation is the comparison of the age group's representation in the general population and the vaccinated population.
It's not difficult to harmonize these measurements. Because the CDC reports the total number of vaccinated individuals for each demographic group, I can calculate that, nationally, the completed rate for those from 40-49 years old is 59.7%, compared to a general rate of 51.2% (which is about 4.5 points lower than the actual figure since there's no age data for about 9% of people who are vaccinated, apparently). So I also know that your within-group rate is 8.5 percentage points higher than the national average. So which figure do I use? There's a big difference between adding 2.0 and 8.5 points.
And perhaps adjusting the state figure with simple addition or subtraction based on demographics in national data is too crude?
We would then repeat this process for gender and race/ethnicity. But not all demographics characteristics are equally predictive. While there is wide variance by age, gender is only moderately different either way you calculate it. This may be accounted for in the fact that the magnitude of the adjustments are smaller.
Let's recall that this is not a random sample, but it's the best we can do and worth the trade-off, I think, for such a massive number of responses.