# Estimating incidence rate for subpopulation from test calibrated on whole population

Formal problem:

I'm given random variables $X\in\{0, 1\}$, $T \in \mathbb{R}$ and $S \in \{0, 1\}$ such that

1) $S$ and $T$ are independent given $X$, i.e. for both $x\in\{0, 1\}$: $$P(S\text{ and }T\in\mathcal{T}|X=x) = P(S|X=x) \cdot P(T\in\mathcal{T}|X=x) \text{ for all sets }\mathcal{T}\subseteq\mathbb{R}$$

Assume I know

2) the joint distribution of $P(X = 1 | T)$ and of $S$, i.e. for all $\mathcal{T}\subseteq\mathbb{R}$, I am given the probabilities $$P(X \text{ and }S | T \in\mathcal{T} ), P(X \text{ and }\bar{S} | T \in\mathcal{T} ), P(\bar{X} \text{ and }S | T \in\mathcal{T} ), P(\bar{X} \text{ and }\bar{S} | T \in\mathcal{T} )$$

I'm interested to know (or estimate, or at least get some rough idea of)

3) $P(X|S)$

I thought it should be easy enough, but I've been banging my head against a wall with this. I'd be grateful for any help.

Context example:

There's a standard test that predicts the chance to have a rare condition X. I notice that people of a small sub-population S appear to have an above average chance for X. I'm interested in estimating that chance using the test that's been calibrated on the whole population.

A first pass would be to take the average of the test predictions for P(X). However, this is likely to underestimate the true incidence rate for X in S:

As X is a rare condition and the test is far from perfect, the test heavily hedges its bets by never predicting a very high probability for X, just like even after a positive HIV test, the chance for a false positive used to be still higher than for a true positive. However, if you measure individuals from the sub-population again and again, and always get the most positive result the test is willing to predict, it's starting to appear quite likely that they probably are all positive.

At least if you assume that the high values of the test are due to some actual marker of X that should be independent of the subpopulation (or the higher results could just be because the test identified the subpopulation). This assumption is expressed in condition 1.

• I understand 1) to mean p(S,T|X) = p(S|X) p(T|X). I'm not sure I understand 2). When you say "joint distribution" do you mean you know something about p(X,S|T)? If not, then what do you mean?
– mef
Commented Jun 13, 2018 at 15:11
• Thanks for your comment. I'll edit my question to be clearer. Yes, this is what I meant with 1), and with 2), I meant that I assume as a given the joint distribution of P(X = 1|T) (which is a T-measurable random variable) and S. Commented Jun 14, 2018 at 8:41

For any event $\mathcal{T}\subseteq\mathbb{R}$ where the denominator is not 0, the desired expression computes as
$$P(X|S) = \frac{P(\mathcal{T}|S) - P(\mathcal{T}|\bar{X})}{P(\mathcal{T}|X) - P(\mathcal{T}|\bar{X})}$$
There always is such an event $\mathcal{T}$ provided that $T$ and $X$ are not independent. If they are independent, there is no hope of using $T$ to get at $P(X|S)$ anyways (in the context example, that would mean the test for X is just useless).