t-test for partially paired and partially unpaired data

Question

An investigator wishes to produce a combined analysis of several datasets. In some datasets there are paired observations for treatment A and B. In others there are unpaired A and/or B data. I am looking for a reference for an adaptation of the t-test, or for a likelihood ratio test, for such partially paired data. I am willing to (for now) to assume normality with equal variance and that the population means for A are the same for each study (and likewise for B).

Answer 1

maybe mixed modelling with patient as random effect could be a way. With mixed modelling the correlation structure in the paired case and the partial missings in the unpaired case could be accounted for.

Answer 2

My first thought was a mixed effects model, but that has already been discussed so I won't say any more on that.

My other thought is that if it were theoretically possible that you could have measured paired data on all subjects but due to cost, errors, or other you don't have all the pairs, then you could treat the unmeasured effect for the unpaired subjects as missing data and use tools like the EM algorithm or Multiple Imputation (missing at random seems reasonble unless the reason a subject was only measured under 1 treatment was related to what their outcome would be under the other treatment).

It may be even simpler to just fit a bivariate normal to the data using maximum liklihood (with the liklihood factored based on the available data per subject), then do a liklihood ratio test comparing the distribution with the means equal vs. different means.

It has been a long time since my theory classes, so I don't know how these compare on optimality.

Answer 3

Here are some thoughts. I basically just arrive to Greg Snow conclusion that This problem has distinct similarities to the Behrens-Fisher problem. To avoid handwaving I first introduce some notations and formalize the hypotheses.

• we have $n$ paired observations $x_i^{pA}$ and $x_i^{pB}$ ($i = 1, \dots, n$);
• we have $n_A$ and $n_B$ unpaired observations $x_i^A$ ($i = 1, \dots, n_A$) and $x_i^B$ ($i = 1, \dots, n_B$);

• each observation is the sum of a patient effect and a treatment effect. The corresponding random variates are

  • $X_i^{pA} = P_i + T_i^A$, $X_i^{pA} = P_i + T_i^B$,
  • $X_i^A = Q_i + U_i^A$, $X_i^A = R_i + V_i^B$

  with $P_i, Q_i, R_i \sim \mathcal N(0,\sigma_P^2)$, and $T_i^\tau, U_i^\tau, V_i^\tau \sim \mathcal N(\mu_\tau, \sigma^2)$ ($\tau = A, B$).

  • under the null hypothesis, $\mu_A = \mu_B$.

We form as usual a new variable $X_i = X_i^{pA} - X_i^{pB}$. We have $X_i \sim \mathcal N(\mu_A - \mu_B, 2\sigma^2)$.

Now we have three groups of observations, the $X_i$ (size $n$), the $X_i^A$ (size $n_A$) and the $X_i^B$ (size $n_B$). The means are

• $X_\bullet\sim \mathcal N(\mu_A - \mu_B, {1\over n} \sigma^2)$
• $X^A_\bullet\sim \mathcal N(\mu_A , {1\over n_A} (\sigma_P^2 + \sigma^2))$
• $X^B_\bullet\sim \mathcal N(\mu_B , {1\over n_B} (\sigma_P^2 + \sigma^2))$

The next natural step is to consider

• $Y = X_\bullet + X^A_\bullet - X^B_\bullet \sim \mathcal N\left( 2(\mu_A-\mu_B), {1\over n} \sigma^2 + \left({1\over n_A}+ {1\over n_B}\right) (\sigma_P^2 + \sigma^2)\right)$

Now basically we are stuck. The three sums of squares give estimations of $\sigma^2$ with $n-1$ df, $\sigma_P^2 + \sigma^2$ with $n_A-1$ df and $n_B-1$ df respectively. The last two can be combined to give an estimation of $\left({1\over n_A}+ {1\over n_B}\right) (\sigma_P^2 + \sigma^2)$ with $n_A+n_B-2$ df. The variance of $Y$ is the sum of two terms, each of which can be estimated, but the recombination is not doable, just as in Behrens Fisher problem.

At this point I think one may plug-in any solution proposed to Behrens Fisher problem to get a solution to your problem.

Answer 4

This is a problem that comes up so much I am kind of surprised that I haven't seen it explicitly tackled.

Well, if you knew the variances in the unpaired and in the paired (which would generally be a good deal smaller), the optimal weights for the two estimates of difference in groups means would be to have weights inversely proportional to the variance of the individual estimates of the difference in means.

The need to estimate variance causes some difficulty (the resulting ratio of variance estimates is F, and I think the resulting weights have a beta distribution, and a resulting statistic is kind of complicated), but since you're considering bootstrapping, this may be less of a concern.

An alternative possibility which might be nicer in some sense (or at least a little more robust to non-normality, since we're playing with variance ratios) with very little loss in efficiency at the normal is to base a combined estimate of shift off paired and unpaired rank tests - in each case a kind of Hodges-Lehmann estimate, in the unpaired case based on medians of pairwise cross-sample differences and in the paired case off medians of pairwise-averages-of-pair-differences. Again, the minimum variance weighted linear combination of the two would be with weights proportional to inverses of variances. In that case I'd probably lean toward a permutation (/randomization) rather than a bootstrap - but depending on how you implement your bootstrap they can end up in the same place.

In either case you might want to robustify your variances/shrink your variance ratio. Getting in the right ballpark for the weight is good, but you'll lose very little efficiency at the normal by making it slightly robust. ---

Some additional thoughts I didn't have clearly enough sorted out in my head before:

This problem has distinct similarities to the Behrens-Fisher problem, but is even harder.

If we fixed the weights, we could just whack in a Welch-Satterthwaite type approximation; the structure of the problem is the same.

Our issue is that we want to optimize the weights, which effectively means weighting is not fixed - and indeed, tends to maximize the statistic (at least approximately and more nearly in large samples, since any set of weights is a random quantity estimating the same numerator, and we're trying to minimize the denominator; the two aren't independent).

This would, I expect, make the chi-square approximation worse, and would almost surely affect the d.f. of an approximation still further.

[If this problem is doable, there also just might turn out be a good rule of thumb that would say 'you can do almost as well if you use only the paired data under these sets of circumstances, only the unpaired under these other sets of conditions and in the rest, this fixed weight-scheme is usually very close to optimal' -- but I won't hold my breath waiting on that chance. Such a decision rule would doubtless have some impact on true significance in each case, but if that effect wasn't so big, such a rule of thumb would give an easy way for people to use existing legacy software, so it could be desirable to try to identify a rule like that for users in such a situation.]

---

Edit: Note to self - Need to come back and fill in details of work on 'overlapping samples' tests, especially overlapping samples t-tests