# Correlation between unpaired data

I have an experiment with 2 mice of 4 different age levels. For each age, I analyse a mouse with method A and the other mouse method B (no mouse is analyse with both method). I want to see whether method A and B is comparable. I have constructed a graph and it shows some correlation. If I use Spearman's correlation on the average, there is strong correlation but not significant because there are only 4 data pairs. Is there any way to take into account all of the data points?

• @AdamO I have more than 2 mice. For method A, I analyse 3 mice on each age level. For method B, I got >5 data (the number is different on each age level). Actually the age correlates to the severity of the disease because age reflects how long we treat the mice. – java Oct 31 '17 at 9:21
• okay thanks for explaining. I feel the question would benefit from better explanation. You must adjust for age or else there will be confounding by indication. – AdamO Oct 31 '17 at 11:08

Try using a regression model instead of a correlation.

You are trying to show the difference between two methods A and B, given the age of a mouse. Define a linear model as follows:

LM <- lm(y ~ method + age)
summary(LM)


Where y are the outcomes for either method, method a dummy variable encoding whether method A or B was used and age the age of the mouse. The summary will include an estimate for the difference between methods (method:B will be the difference between A and B). The intercept will be that of method A.

If you believe the age has a different effect, depending on the method used, include an interaction term:

LM.with.int <- lm(y ~ method * age)


Since these are only three variables, you could visualize the entire dataset using a scatter plot with distinct colours/characters for age:

plot(y ~ x, col = age) # colour
plot(y ~ x, pch = age) # character

• Is it the way to use R to calculate the regression? – java Oct 27 '17 at 8:17
• This is R code, which is arguably the most popular here. But you can also do regression in other programs. – Frans Rodenburg Oct 27 '17 at 8:19
• It seems like my data has monotonic relationship but non-liner (the value goes up then steady). Is it the same as linear with interaction on age? – java Oct 27 '17 at 8:32
• You could try a logarithmic transformation of age: log(age). – Frans Rodenburg Oct 27 '17 at 8:53
• @java no what you are describing is not (necessarily) an interaction. Age is just a blocking factor, but you are trying to model an experimental effect. It may simply be a curvilinear trend. How is it possible you have two experimental conditions but are observing non-linear trends? – AdamO Oct 27 '17 at 13:25

If your study is underpowered, you must simply accept that you do not have enough mice to draw inference. This does not mean you should assess the adequacy of age as a predictor of the outcome, and assess the validity of paired analysis as an appropriate analytic approach. I would underscore, however, that if your research is disseminated like in a journal article, you should describe in detail all statistical tests that you tried. You had a prespecified analysis and you are now departing from it due to lack of statistical significance. This is p-hacking, and it will inflate the type 1 error rate.

It sounds like your sample comprises 8 randomly sampled mice, block randomized on age 1:1 to experimental conditions A and B. A loss of precision suggests that age is not (clinically) significantly prognostic of the outcome. While a paired analysis reduces the effective $n$ of the sample, the residual variability of the paired-differences should be halved, if not more, by design, thus improving efficiency... but only when paired clusters have a very high intracluster correlation. Block-randomized studies may or may not adjust for the blocking factor in some explicit way (multivariate analysis, fixed or random effects) or implicit way (conditional inference vis-a-vis paired t-test). Choosing not to adjust for the blocking factor relies on having balanced design for unbiased estimation of treatment effect and correct inference. You have that. So good! At this point a number of regression models should come to mind: a simple t-test, a regression model, or a mixed effects model.

I might suggest you can obtain powerful inference by using a simple permutation test. They perform well in small samples, the p-value does not depend on any sample characteristics even with such small $n$, like having approximate normality.