How to combine state-level COVID-19 vaccination rates with national demographic data at the individual level?

Question

I’m investigating possible correlations between the COVID-19 vaccination rate in the United States and the results of a long-running survey of scientific personality traits like "Agreeableness" and "Conscientiousness." While the survey does not ask about vaccination status, it does (optionally and anonymously) ask for self-reported state of residence, age, gender and/or race/ethnicity. This was all conducted under IRB supervision and limited to those age 18 or over. It is a public survey anyone can take, not a poll, so it is not a representative sample.

My challenge is to estimate each respondent's odds of being fully vaccinated based on his or her location and demographic info, taking the state-level vaccination rate and somehow adjusting it with national data on vaccination status by age, gender and race/ethnicity, all from the CDC.

TL;DR: How can I adjust one probability, based on a smaller population (state of residence) with extra information (national variance by age, gender and race/ethnicity) to make the estimate as precise as possible?

(I suspect this is the same exercise a car insurance company conducts when choosing the cost of a policy, using both traffic and crime data in your locality and the odds that other people in your demographic cohort are more or less likely to get in accidents.)

My instinct is to start with the completed vaccination rate in each respondent's state and then simply add or subtract to that value based on how many percentage points more or less likely that individual's age group, gender and race/ethnicity group is to be vaccinated compared to the total population on a national level. But even if this is the correct instinct, it gets tricky because there are two ways of looking at the predictive power of a demographic characteristic.

For example, if I know you live in Delaware and are age 18 or older, I know from CDC data (updated daily) that there's a 68.5% chance you're fully vaccinated (both doses of Pfizer/Moderna or one of J+J). If you're age 65 or over, the rate jumps to 87.3%. That's the only age breakdown the CDC offers at a state level.

But let's say I also know you're 44 years old. The way the CDC reports demographic data is by simply publishing both the percentage of fully vaccinated people in each age group as well as the Census estimate for the percentage of the total population in that age group. So I know that, while only 12.2% of the population is between ages 40 and 49, 14.2% of fully vaccinated individuals are in this age range. I suspect the reason the CDC reports the data in this way is that age is only available for 91% of those vaccinated.

So how do I adjust your base rate of 68.5% with the fact that, across the entire population, those between 40 and 49 years old are over-represented among the fully vaccinated population? The most obvious answer is to add 2.0 percentage points, the difference between 12.2% and 14.2%.

But this may be foolish, because we're dealing with two different types of measurements. The state-level rate is just the percentage of people in the 18+ bracket who are fully vaccinated in each state, regardless of how many of them there are--thus allowing one to easily compare Wyoming to California. Meanwhile, the more precise age-level data for the nation is the comparison of the age group's representation in the general population and the vaccinated population.

It's not difficult to harmonize these measurements. Because the CDC reports the total number of vaccinated individuals for each demographic group, I can calculate that, nationally, the completed rate for those from 40-49 years old is 59.7%, compared to a general rate of 51.2% (which is about 4.5 points lower than the actual figure since there's no age data for about 9% of people who are vaccinated, apparently). So I also know that your within-group rate is 8.5 percentage points higher than the national average. So which figure do I use? There's a big difference between adding 2.0 and 8.5 points.

And perhaps adjusting the state figure with simple addition or subtraction based on demographics in national data is too crude?

We would then repeat this process for gender and race/ethnicity. But not all demographics characteristics are equally predictive. While there is wide variance by age, gender is only moderately different either way you calculate it. This may be accounted for in the fact that the magnitude of the adjustments are smaller.

Let's recall that this is not a random sample, but it's the best we can do and worth the trade-off, I think, for such a massive number of responses.

Answer 1

I found a much more straightforward method to calculate this than what is in my other answer.

To begin, from the official statistics, we get the percentage of people vaccinated in Delaware, which is 56.6%. Let $D$ be the total population of Delaware. Then there are $0.566 \cdot D$ vaccinated persons in Delaware.

The number of people in the US in the age group 40-49 is 12.2%. But they make up 14.2% percent of the people vaccinated. Let's assume these percentages hold in Delaware as well.

Then the total number of people aged 40-49 living in Delaware is $0.122\cdot D$. And the number of people vaccinated aged between 40-49 is 14.2% of vaccinated subjects. So the final percentage is

$$ \frac{0.142 \cdot 0.566 \cdot D}{0.122 \cdot D} = \frac{.142 \cdot .566}{0.122} \approx 65.9\% $$

Answer 2

I don't think you can add probabilities like that, but if you convert to log odds first, you can combine the two together. You will need to assume that the distribution of vaccinations in the age groups is the same in Delaware as in the rest of the US. The question looks straightforward but it took me a while to come up with a solution, which I posted as a blogpost.

Summarizing;

The log odds of the us population being vaccinated is 0.245.

The coefficient for living in Delaware is 0.020. This is calculated from the difference between 56.6% of Delaware population being vaccinated, and 56.1% of the whole population in the US.

The coefficient for the age group 40-49 is 0.387.

Assuming independence you can add these log odds to obtain a log odds of 0.652 for a person in Delaware, aged 40-49 being vaccinated. This corresponds to a percentage of 65.8%.

In this answer, I simplified by using the Delaware overall statistics, instead of trying to adjust based on the 18-65 group. That is also possible, but slightly more complicated.

Answer 3

Another approach is to use bayes theorem. Assuming independence between age and state (i.e. assuming similar age distribution between states) we are after

$P (F | A, S)$ - probability of being fully vaccinated given state and age.

where we know only $P(F | A), P (F | S)$

$P (F | A, S) = \frac{P(A, S | F)P(F)}{P(A,S)} = \frac{P(S|F)P(A|F)P(F)}{P(A) P(S)} = \frac{P(F|S)P(S)P(A|S)P(A)P(F)} {P(F)P(F)P(A)P(S)} $

Finally we get:

$P (F | A, S) = \frac{P(F|S)P(F|A)}{P(F)}$

Which also have intuitive interpretation, take the probability of being fully vaccinated in specific state and "fix" is with the ratio of being fully vaccinated in a specific age over probability of being fully vaccinated.