Do you have any suggestions, which test I should use in order check if the relative change is statistically different than 0. I am sure that I should check the relative (fold) changes.

Specifically, for each observation I have two measurements: first and second. I would like to check if $\rm \frac{measurement_{second} - measurement_{first}}{mesurement_{first}}$ is different than 0.

In this question: The result of absolute change and relative change in t test is different, it is suggested to make use of logarithm of change. How to use these logarithms? Something like that:

I transform data: $\rm \log (mesurement_{second}) - \log (mesurement_{first})$ and use paired t-test? What other tests I can use after such transformation?

Are there tests that tackle my problem directly without transformation of data?

  • $\begingroup$ One way to understand the relationship between the log transformation of two values $x_1$ and $x_2$ and their relative error $e=\frac{x_2-x_1}{x_1}$ is to note that $\log(x_2)-\log(x_1) = \log(\frac{x_2}{x_1}) = \log(1 + e) = e + O(e)^2,$ showing that when $|e|\ll 1$ (which frequently is the case unless changes are dramatic), a difference of logs basically is the relative error. $\endgroup$ – whuber May 7 '14 at 18:23
  • $\begingroup$ That is useful. However the part $O(e)^2$ makes me quite uncomfortable, because I am not sure, how would this affect the test results. So should we test $\log (x_2) - \log(x_1) = 0$ vs. $<>0$? With whatt test can I carry on? $\endgroup$ – kuba May 7 '14 at 18:43

First you should decide if you are interested in the mean of the relative changes or the relative change of the means (or substitute median or other summary for mean). We don't worry about this with the paired t-test since the difference of the means is algebraically equal to the mean of the differences. But this is not the case for relative change and other comparisons.

I would suggest doing a permutation test. Your title says "paired" so you can do a paired test:

  1. compute the statistic of interest for the original data
  2. within each pair randomly choose to flip the order or keep it the same
  3. recompute the statistic of interest with the permuted (flipped) data
  4. redo 2 and 3 a bunch of times (keeping track of all the computed stats).

The p-value is the proportion of permuted statistics that are as or more extreme than your original.

This can be done without any transformations, with mean or median, etc. Whatever makes the most sense to you.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.