I have an experiment with 2 mice of 4 different age levels. For each age, I analyse a mouse with method A and the other mouse method B (no mouse is analyse with both method). I want to see whether method A and B is comparable. I have constructed a graph and it shows some correlation. If I use Spearman's correlation on the average, there is strong correlation but not significant because there are only 4 data pairs. Is there any way to take into account all of the data points?

  • $\begingroup$ @AdamO I have more than 2 mice. For method A, I analyse 3 mice on each age level. For method B, I got >5 data (the number is different on each age level). Actually the age correlates to the severity of the disease because age reflects how long we treat the mice. $\endgroup$ – java Oct 31 '17 at 9:21
  • $\begingroup$ okay thanks for explaining. I feel the question would benefit from better explanation. You must adjust for age or else there will be confounding by indication. $\endgroup$ – AdamO Oct 31 '17 at 11:08

Try using a regression model instead of a correlation.

You are trying to show the difference between two methods A and B, given the age of a mouse. Define a linear model as follows:

LM <- lm(y ~ method + age)  

Where y are the outcomes for either method, method a dummy variable encoding whether method A or B was used and age the age of the mouse. The summary will include an estimate for the difference between methods (method:B will be the difference between A and B). The intercept will be that of method A.

If you believe the age has a different effect, depending on the method used, include an interaction term:

LM.with.int <- lm(y ~ method * age)

Since these are only three variables, you could visualize the entire dataset using a scatter plot with distinct colours/characters for age:

plot(y ~ x, col = age) # colour  
plot(y ~ x, pch = age) # character
| cite | improve this answer | |
  • $\begingroup$ Is it the way to use R to calculate the regression? $\endgroup$ – java Oct 27 '17 at 8:17
  • $\begingroup$ This is R code, which is arguably the most popular here. But you can also do regression in other programs. $\endgroup$ – Frans Rodenburg Oct 27 '17 at 8:19
  • $\begingroup$ It seems like my data has monotonic relationship but non-liner (the value goes up then steady). Is it the same as linear with interaction on age? $\endgroup$ – java Oct 27 '17 at 8:32
  • $\begingroup$ You could try a logarithmic transformation of age: log(age). $\endgroup$ – Frans Rodenburg Oct 27 '17 at 8:53
  • $\begingroup$ @java no what you are describing is not (necessarily) an interaction. Age is just a blocking factor, but you are trying to model an experimental effect. It may simply be a curvilinear trend. How is it possible you have two experimental conditions but are observing non-linear trends? $\endgroup$ – AdamO Oct 27 '17 at 13:25

If your study is underpowered, you must simply accept that you do not have enough mice to draw inference. This does not mean you should assess the adequacy of age as a predictor of the outcome, and assess the validity of paired analysis as an appropriate analytic approach. I would underscore, however, that if your research is disseminated like in a journal article, you should describe in detail all statistical tests that you tried. You had a prespecified analysis and you are now departing from it due to lack of statistical significance. This is p-hacking, and it will inflate the type 1 error rate.

It sounds like your sample comprises 8 randomly sampled mice, block randomized on age 1:1 to experimental conditions A and B. A loss of precision suggests that age is not (clinically) significantly prognostic of the outcome. While a paired analysis reduces the effective $n$ of the sample, the residual variability of the paired-differences should be halved, if not more, by design, thus improving efficiency... but only when paired clusters have a very high intracluster correlation. Block-randomized studies may or may not adjust for the blocking factor in some explicit way (multivariate analysis, fixed or random effects) or implicit way (conditional inference vis-a-vis paired t-test). Choosing not to adjust for the blocking factor relies on having balanced design for unbiased estimation of treatment effect and correct inference. You have that. So good! At this point a number of regression models should come to mind: a simple t-test, a regression model, or a mixed effects model.

I might suggest you can obtain powerful inference by using a simple permutation test. They perform well in small samples, the p-value does not depend on any sample characteristics even with such small $n$, like having approximate normality.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.