Using non-random sample to make population estimates

Question

The health screening for immigration to the US includes a mandatory chest x-ray screening for all adults, with the aim of detecting signs of tuberculosis. If the x-ray indicates the possibility of TB, the applicant then submits sputum samples for culturing. That's a lot of x-rays - and many of them don't result in positive cultures, especially among older applicants (whose lungs can be funky for many reasons).

In a study, 1,500 applicants (out of 24,000 during the study period) were given a new TB diagnostic in addition to the standard screening process.* People with chest x-rays indicative of TB were oversampled to ensure a fair number of applicants with culture-positive sputa. Now, the researchers would like to know (for example): if we had performed this additional diagnostic on all applicants, how many people would have tested positive? Of those, how many would have had a TB-indicative x-ray? Of those, how many would have had positive cultures? Demographic, x-ray, and culture results are known for all applicants; new-diagnostic result is only known for study participants.

Two approaches for this study have been proposed.

1. Build a logistic regression on the sample and use to predict for all applicants. There are concerns that the standard errors will be too small, as age, x-ray result, and culture result are quite correlated.
2. Pseudo-bootstrap. Sample 24,000 results from the participants' results (with replacement), assigning a sampling probability to each participant based on the number of all applicants with matching characteristics (demographics, x-ray result, culture result). From the resample, calculate desired quantities. Repeat, calculate confidence intervals for the estimates.

Which of these two would you use, if either? Why?

I'm happy to clarify any questions. Thanks for your help!

*Some study details changed to protect the word count.

Answer 1

The $24,000$ applicant population was divided into two strata

1. The $N_1$ applicants who had chest x-rays indicative of TB
2. The $N_2=24,000-N_1$ who had chest x-rays not indicative of TB

A stratified random sample was taken. This included $n_1$ participants with x-rays indicative of TB and $n_2=1,500-n_1$ participants with x-rays not indicative of TB. The sampling weights for these two strata are $w_1=\frac{N_1}{n_1}$ and $w_2=\frac{N_2}{n_2}$.

Within the first of these strata, $x_1$ tested positive with the new method. Within the second of these strata, $x_2$ tested positive. The estimate of how many of the $24,000$ applicants would have tested positive with the new method is thus the weighted sum $T=w_1\times x_1+w_2\times x_2$.

Considering the total to be taken from a normal distribution, we can estimate its standard error by adding the estimated variances within each stratum and applying a finite population correction.

You can additionally stratify by the demographic data. For example, if you stratified by binary sex as well, you would have four strata, each with their own weights.

1. Men with chest x-rays indicating the possibility of TB
2. Women with chest x-rays indicating the possibility of TB
3. Men with chest x-rays not indicating the possibility of TB
4. Women with chest x-rays not indicating the possibility of TB

Depending on the sampling process, you may call this post-stratification.

You can read more about this in a book on sampling, such as in chapter 6 of Levy & Lemeshaw's Sampling of Populations.

This answer seems too simple, so I suspect that I am missing something. Do tell me if I am.

Answer 2

You have measured 3 variables, X-ray (pos or neg), sputum (pos, neg or not tested), new diagnostic (pos, neg or not tested), and you may have 9 outcome groups:

  group X-ray sputum  new
a     1     +      +    +
a     2     +      +    -
a     3     +      +   nt
b     4     +      -    +
b     5     +      -    -
b     6     +      -   nt
c     7     -     nt    +
c     8     -     nt    -
c     9     -     nt   nt

You told us that the total number of applicants $N=\sum_{g=1}^9{n_g}=24000$, and that $n_1+n_2+n_4+n_5+n_7+n_8=1500$. You also told that the covariate age was available for everyone and correlated with X-ray and sputum.

Assuming that you took independently 3 random samples from (a) the X-ray +, sputum + groups (1, 2, 3), (b) the X-ray +, sputum - groups (4, 5, 6) and (c) the X-ray -, sputum nt groups (7, 8, 9), you can fit logistic models within groups (a), (b) and (c) predicting new relying on the covariate age. Using the three logistic models you can predict the probability of being new + for each individual in groups 3, 6 and 9. Adding up these probabilities gives a point estimate of the number of new + applicants in groups 3, 6 and 9. (This is similar to your method 1.)

A method to assess the uncertainty in these estimates may be multiple imputation. Let's see its details on the X-ray +, sputum + groups (1, 2, 3): Using multiple imputation on the dataset containing groups 1, 2 and 3 (with the logistic model of age as predictor) to impute the missing new values throughout group 3 should give you realistic new values in group 3. Taking multiple imputed datasets and counting the new + cases in each you will both cover the uncertainty stemming from the binomial distribution after logistic regression and the uncertainties (and covariance) of the coefficients estimated in the logistic model. A large number of imputations will minimise the Monte Carlo error.

Your method 2 neglects that you want to do prediction on the finite population of your specific 24000 applicants.