# Longitudinal comparison of two distributions

I have the test results of a blood test administered to 2500 people four times at six-month intervals. The results primarily consist of two measures of immune response - one in the presence of certain tuberculosis antigens, one in the absence. Currently, each test evaluates to either positive or negative based on the difference between the antigen response and the nil response (with the idea being that if your immune system responds to TB antigens, you've likely been exposed to the bacterium itself at some point). In essence, the test supposes that a non-exposed individual's distributions of nil and TB responses should be basically identical, whereas a person with TB exposure will have TB responses drawn from a different distribution (of higher values). Caveat: the responses are very, very non-normal, and values clump at both the natural floor and the instrument-truncated ceiling.

However, it's seems pretty clear in this longitudinal setting that we're getting "false positives" (no actual gold standard for latent tuberculosis, I fear) that are caused by (typically small) fluctuations in the antigen and nil responses. While this might be hard to avoid in some situations (you may only get one chance to test someone), there are many situations in which people are routinely tested for TB every year or so - in the US, this is common for healthcare workers, the military, homeless people staying at shelters, and so on. It seems a shame to ignore prior test results because the extant criteria happen to be cross-sectional.

I think that what I'd like to do is what I crudely conceive of as longitudinal mixture analysis. Much like the cross-sectional criteria, I'd like to be able to estimate the probability that an individual's TB and nil responses are drawn from the same distribution - but have that estimate incorporate prior test results, as well as information from the sample as a whole (e.g., can I use the sample-wide distribution of within-individual variabilities to improve my estimates of a specific individual's distribution of nil or TB?). The estimated probability would need to be able to change over time, of course, to account for the possibility of new infection.

I've gotten myself totally twisted around trying to think about this in unusual ways, but I feel like this conceptualization is as good as any I'm going to come up with. If something doesn't make sense, please feel free to ask for clarification. If my understanding of the situation seems wrong, please feel free to tell me. Thank you so much for your help.

In response to Srikant: It's a case of latent classification (TB-infected or not) using the two continuous (but non-normal and truncated) test results. Right now, that classification is done using a cutoff (in its simplified form, TB - nil > .35 -> positive). With test results presented as (nil, TB, result), the basic archetypes* are:

Probable Negative: (0.06, 0.15, -) (0.24, 0.23, -) (0.09, 0.11, -) (0.16, 0.15, -)
Probable Positive: (0.05, 3.75, +) (0.05, 1.56, +) (0.06, 5.02, +) (0.08, 4.43, +)
Wobbler: (0.05, 0.29, -) (0.09, 0.68, +) (0.08, 0.31, -) (0.07, 0.28, -)

The positive on the second test for the Wobbler is pretty clearly an aberration, but how would you model that? While one line of my thinking is to estimate the "true difference" between TB and nil at each time point using a repeated-measures multilevel model, it occurred to me that what I really want to know is if the person's nil response and TB response are drawn from the same distribution, or if their immune system recognizes the TB antigens and activates, producing an increased response.

As for what could cause a positive test other than infection: I'm not sure. I suspect it's typically just within-person variation in results, but there's certainly a possibility of other factors. We do have questionnaires from each time point, but I haven't looked into those too much yet.

*Fabricated but illustrative data

• Oh, and feel free to retag - my browser isn't working with the auto-suggest, so I'm having a hard time seeing what's out there. – Matt Parker Sep 24 '10 at 19:55
• Is your dependent variable continuous or discrete? Or, perhaps, the underlying test result is continuous and it is converted into a discrete answer (i.e., 'positive', 'negative') depending on some cut-off? Could you also clarify why an individual would flip from negative to positive despite not being exposed to TB? A specific example (with some numbers thrown in) of such a flip may help. – user28 Sep 29 '10 at 19:21
• The examples are really helpful to visualize the data. Another question regarding your caveat: "the values clump at the floor and the ceiling and that the data are not normal." Can you tell me if (a) the data on the lower end of the scale look normal and (b) the data on the upper end of the scale look normal? – user28 Sep 30 '10 at 0:23
• Note: I apparently missed the deadline to actually award the bounty, so I'm setting up another so that I can properly reward Srikant for his help. More answers are always welcome, but the bounty is for him. – Matt Parker Oct 6 '10 at 23:13

This is not a complete answer but I hope it gives you some ideas as to how to model the situation in a coherent manner.

Assumptions

1. The values at the lower end of the scale follow a normal distribution truncated from below.

2. The values at the upper end of the scale follow a normal distribution truncated from above.

(Note: I know that you said that the data is not normal but I am assuming that you are referring to the distribution of all the values whereas the above assumptions pertain to the values at the lower and the upper end of the scale.)

3. A person's underlying state (whether they have TB or not) follows a first-order markov chain.

Model

Let:

1. $D_i(t)$ be 1 if at time $t$ the $i^\mbox{th}$ person has TB and 0 otherwise,

2. $RTB_i(t)$ be the test response to the TB test at time $t$ of the the $i^\mbox{th}$ person,

3. $RN_i(t)$ be the test response to the NILL test at time $t$ of the the $i^\mbox{th}$ person,

4. $f(RN_i(t) | D_i(t)=0) \sim N(\mu_l,\sigma_l^2) I(RN_i(t) > R_l)$

5. $f(RN_i(t) | D_i(t)=1) \sim N(\mu_l,\sigma_l^2) I(RN_i(t) > R_l)$

Points 4 and 5 capture the idea that a person's response to the NILL test is not dependent on disease status.

6. $f(RTB_i(t) | D_i(t)=0) \sim N(\mu_l,\sigma_l^2) I(RTB_i(t) > R_l)$

7. $f(RTB_i(t) | D_i(t)=1) \sim N(\mu_u,\sigma_u^2) I(RTB_i(t) < R_u)$

8. $\mu_u > \mu_l$

Points 6, 7 and 8 capture the idea that a person's response to the TB test is dependent on disease status.

9. $p(t)$ be the probability that a person's catches TB during the 6 months preceding time $t$ given that they were disease free during the previous test period. Thus, the state transition matrix would like the one below:

$\begin{bmatrix} 1-p(t) & p(t) \\ 0 & 1 \end{bmatrix}$

In other words,

$Prob(D_i(t)=1 | D_i(t-1) = 0) = p(t)$

$Prob(D_i(t)=0 | D_i(t-1) = 0) = 1-p(t)$

$Prob(D_i(t)=1 | D_i(t-1) = 1) = 1$

$Prob(D_i(t)=0 | D_i(t-1) = 1) = 0$

$\hat{D}_i(t) = \begin{cases} 1, & RTB_i(t) - RN_i(t) \ge 0.35 \\ 0, & otherwise \end{cases}$

However, as you see from the structure of the model you can actually parameterize the cut-offs and change the whole problem to that of what should be your cut-offs to accurately diagnose patients. Thus, the wobbler problem seems to be more an issue with your choice of cut-offs rather than anything else.

In order to choose the 'right' cut-offs, you can take historical data about patients definitively identified as having TB and estimate the resulting parameters of the above setup. You could use some criteria such as number of patients correctly classified as having TB or not as a metric to identify the 'best' model. For simplicity, you could assume that $p(t)$ to be a time invariant parameter which seems reasonable in the absence of epidemics etc.

Hope that is useful.

• Thanks, Srikant! Sorry, I somehow missed your comment earlier. The upper cluster is really just a spike right at the ceiling - there's no variability there except for the long stretch of uniformity that links it to the lower distribution, which is basically as you describe. It'll take me some time to parse out your answer (especially since I'm stuck in IE and can't see the LaTeX properly right now), but I really appreciate your dedication to this odd little question. – Matt Parker Oct 6 '10 at 23:09

Tricky Matt, as many real-world stats problems are!

I would start be defining your study aims/objectives.

Without knowing the true status of the subjects it will be hard to define the probability distributions for the TB+ and TB- test. Do you have questionairres regarding previous TB infection (or better, medical histories). Also I still test TB+ due to an immunisation in childhood - several decades ago - so previous immunisations need to be considered.

It seems to me your intrinsic question is: Does repeated TB testing affect test outcome?

It would be worth getting a copy of Peter Diggle's Analysis of Longitudinal Data.

Do some exploratory data analysis, particularly scatter plot matrices of the nil-test results at each time versus each other, and the TB test results at each time versus each other; and the TB vs nil scatter plots (at each time). Also take the differences (TB test - Nil test) and do the scatter plot matrices. Try transformations of the data and redo these - I imagine log(TB) - log(Nil) may help if the TB results are very large relative to Nil. Look for linear relations in the correlations structure.

Another approach would be to take the defined test result (positive/ negative) and model this logitudibnally using a non-linear mixed effects model (logit link). Do some individuals flip between testing TB+ to TB- and is this related to their Nil test, TB test, TB - Nil or some transformation of test results?

• Thanks for your answer. Regarding not knowing the true status: we do have extensive questionnaires and are well aware of the BCG vaccine issue with the skin test - in fact, these blood tests are supposed to resolve that issue because they use a different set of antigens than the PPD you're used to. That's almost a separate question, however, and one we're going to be working on a bit later - right now, my interest is in making this test 'longitduinally aware'. – Matt Parker Sep 25 '10 at 14:50
• ... especially because some individuals do flip from negative to positive, and that's often a product of their typical nil and TB results making small fluctuations - nil down a bit, TB up a bit, and suddenly they're positive. Next test, they've gone back to being negative. I can see that as I review individual results, but I'm not sure how to appropriately incorporate my intuition into a model. – Matt Parker Sep 25 '10 at 14:54
• Finally, while I have tried taking the log results, that doesn't seem to be sufficient to get them even close to normality. They are very, very skewed, and the truncation at the high end further complicates this by adding a noticeable blob of density at the ceiling. Interestingly, however, the sample-wide nil and TB result distributions are quite similar, with the only difference being that that blob on the ceiling is much larger for the TB results. – Matt Parker Sep 25 '10 at 14:57
• Thanks for taking the time to read and answer this beast of a question! – Matt Parker Sep 25 '10 at 14:57