Mann-Whitney with repeated measures on only *some* participants

Question

I've been working on the stats for a paper which involves comparison of various pairs of groups on a particular score that is heavily skewed (and so I don't feel it would be well suited to comparison of means). The scientists I'm working with are rather keen to have p-values alongside my lovely CIs. No problem, in general; chuck in a Mann-Whitney U. The issue arises where I have one comparison they'd like to make, which is partially repeated measures: around half of participants have contributed to scores in both conditions (while the rest have only contributed scores in 1).

I'm a bit stuck on how to test this? It would be nice from my point of view if it could be a test on the Mann-Whitney U, as that would sit well with my other tests (and CIs), and I like how U captures the probability of superiority (which my CIs are also aimed at). But if that's not possible, I could consider alternatives.

I've presently done it by simulating U, using the correlation matrix of the data as the covariance matrix for simulated normal data (so that both cases come from the same distribution but with correlation) and then deleting cases to capture the non-repeated cases. I feel like this is probably "close enough", but feels hacky and of course is incorporating the observed correlation etc into the null. The p-values are tiny however you calculate it, so it doesn't feel that important, but I'm afraid of it getting ripped up later down the line.

Can anyone suggest a better alternative? Would a permutation work in this case? I'm struggling to wrap my head around how they would apply to repeated data.

---

Clearer understanding of the data:

The academics I'm working with have put out a questionnaire to dog/cat owners. So, the data is scale scores (sum of a large number of Likert items), answered once for a dog that they own, and once for a cat that they own. Those respondents who own both will answer both, those that own only one, will answer just one. I'd like to compare the total scores between dogs and cats, without removing those scores where only one is owned. There is just one total score for each.

Answer 1

So far just some ideas. According to this paper, also referenced in this book by Frank Harrell, the Mann-Whitney test is equivalent (in some sense) to ordinal logistic regression. In the case of two groups, that is the Wilcoxon test is equivalent to logistic regression. This is somewhat approximate, the equivalence is not with the likelihood ratio test which are usually used, but with the score test.

An idea is to exploit this in the case with repeated measures (since repeated measures modeled via mixed models do not have problems with missing observations in some groups.) So use a logistic repeated measures model, the usual one if you have two groups, or proportional odds LR in case of more groups. I can try to flesh that more out if you can give some more details.

As a test of the equivalence, I did some simulations in R:

k <- 50
set.seed(7*11*13)#My public seed
x <- rep(0:1, each=k)
B <- 1000
result <-t(replicate(B, {
    y <- rbinom(2*k, 1, rep(c(0.3, 0.55), each=k))
    pval.wilcox <- wilcox.test(y  ~ x)$p.value
    pval.glm    <- anova(glm(y  ~ x, family=binomial), test="Chi")$`Pr(>Chi)`[2]
    c(pval.wilcox=pval.wilcox, pval.glm=pval.glm)
    }, simplify=TRUE))

Looking at the results, the p-value from wilcox.test is newer smaller than the p-value from logistic regression, but the largest difference is $-0.005$. So this seems to work well!