I've been working on the stats for a paper which involves comparison of various pairs of groups on a particular score that is heavily skewed (and so I don't feel it would be well suited to comparison of means). The scientists I'm working with are rather keen to have p-values alongside my lovely CIs. No problem, in general; chuck in a Mann-Whitney U. The issue arises where I have one comparison they'd like to make, which is partially repeated measures: around half of participants have contributed to scores in both conditions (while the rest have only contributed scores in 1).
I'm a bit stuck on how to test this? It would be nice from my point of view if it could be a test on the Mann-Whitney U, as that would sit well with my other tests (and CIs), and I like how U captures the probability of superiority (which my CIs are also aimed at). But if that's not possible, I could consider alternatives.
I've presently done it by simulating U, using the correlation matrix of the data as the covariance matrix for simulated normal data (so that both cases come from the same distribution but with correlation) and then deleting cases to capture the non-repeated cases. I feel like this is probably "close enough", but feels hacky and of course is incorporating the observed correlation etc into the null. The p-values are tiny however you calculate it, so it doesn't feel that important, but I'm afraid of it getting ripped up later down the line.
Can anyone suggest a better alternative? Would a permutation work in this case? I'm struggling to wrap my head around how they would apply to repeated data.
Clearer understanding of the data:
The academics I'm working with have put out a questionnaire to dog/cat owners. So, the data is scale scores (sum of a large number of Likert items), answered once for a dog that they own, and once for a cat that they own. Those respondents who own both will answer both, those that own only one, will answer just one. I'd like to compare the total scores between dogs and cats, without removing those scores where only one is owned. There is just one total score for each.