# Do transformations caricaturize the data? Are corresponding outcomes of hypothesis tests artefacts?

I am getting different significance outcomes when using non-parametric tests on the raw data versus log-transforming the data and then applying parametric tests. Which one is more valid?

I have Machine 1 (the reference) and Machine 2 (the experimental) which count the number of dots on a screen. The tests on the machine were run in parallel and each output can thus be considered paired for comparison. I want to test if there is significant difference in counts made between the machine.

Given below is the dataset (in R):

df<-data.frame(
Machine1 = c(61,155,49,433,56,44,30,26,57,137,131,23,137,20,30,28,30,246,39,267,76,40,85,30,49,430,97,252,341,91,100,66,30,32,145,18,69,119,261,14,30,50,110,47,12,7,162,24,30,314,40,31,69,300,103,549,66,20,148,120,216,61,247,101,41,32,30,89,32,18,46,161,31),
Machine2= c(62,165,44,453,56,120,165,30,82,140,151,49,108,11,14,23,30,686,33,552,67,33,75,26,38,450,84,250,329,81,475,57,25,102,131,18,67,108,240,11,27,48,261,54,12,8,160,32,22,510,72,28,400,150,90,440,65,20,115,186,200,350,507,149,35,21,26,198,29,40,76,145,36),
)


The outcomes for Machine 1 and Machine 2 are not normal as evidenced by the Shapiro-Wilk's test:

Machine1: W=0.75554, p-value=1.087e-9
Machine2: 0.75956, p-value = 1.356e-09


1. Using non-parametric approach:

Now, I could perform a Wilcoxon test using the code:

wilcoxon.test(x=df$$Machine1, y=df$$Machine2, paired=TRUE, exact=FALSE)


and find that the differences are insignificant:

V=1002, p-value=0.2974


Conclusion 1: There is no significant difference in the number of dots counted by the machines

2. Using transformations + parametric tests

When the data is log-transformed using the code:

df.log<-log10(df)


The corresponding Shapiro-Wilk's test outcomes are:

W = 0.97656, p-value = 0.1912
W = 0.97601, p-value = 0.1778


So I can proceed with a Student's paired t-test with the code:

t.test(x=df.log$$Machine1, y=df.log$$Machine2, paired = TRUE)


And find that differences are now significant:

t = -2.5372, df = 72, p-value = 0.01334


Conclusion 2: There is significant difference in the number of dots counted by the machines

My basic question/doubt:

The raw data was the output given by the machines themselves. By transforming them to fit assumptions of a parametric test, we are getting a set of related, but different numbers with a very different profile. These number-sets obviously behave differently when subjected to statistical tests.

So, is the data transformation approach generally an invalid one?

If you were in a situation where you want to spend money and buy Machine 2 (assuming it worked just like Machine 1), which approach would you use to arrive at a decision?

• Chemical theory strongly points to the fact that logarithms of concentrations are meaningful and relevant. Should chemists modify their theories and calculations because their instruments record concentrations rather than the logarithms?? I think not. You should not be a slave to the method that is employed to write down a result. Any number is merely a name -- an identifier -- for a quantity. An insightful analysis will look beyond mere names and seek meaningful relationships. If those are revealed by changing the names, what would be the problem with that?
– whuber
Commented Jan 20, 2023 at 16:41

I think the key difference is that a transformation loses no information, insofar as given the transformed values and the transformation you can in principle recover the original data. (Small print: transformations here are assumed invertible.) But that isn't true of ranking. Ranking always loses some information. The selling point for ranking is whenever it doesn't lose much information and that you shouldn't want to apply methods if the associated ideal conditions (most people say "assumptions" here) are not well approximated.

An analogy I like to use is that between a transformation in statistics and a map projection (which is also a transformation). A map with some chosen projection shows some part of the Earth in a two-dimensional space, and it's manifest that a map projection has to distort the data to some degree -- otherwise we have to consult globes, not phones, laptops, maps or atlases, and globes have limitations too. But a caricature, which is what any map is to some degree, can still be faithful to the data in a strong sense.

My view, and I think it is widely shared, is that applying a transformation when it does what you want is always better use of the information than resorting to a non=parametric test, which usually changes the question any way. If the concern is with mean differences, or differences between means, non=parametric tests don't really get at that, and it's wishful thinking to pretend that they do.

Some experimentation with your data suggests that logarithms are a good scale to work on. I wouldn't over-emphasise testing, and key graphs to draw include

1. Normal quantile plots of original data on logarithmic scale and of difference, meaning log(Machine1) - log(Machine2).

2. Scatter plot of Machine1 and Machine2 on log scales.

3. Scatter plot of difference versus mean, meaning (log(Machine1) + log(Machine2))/2.

• Not all transformations are loss-less, though perhaps the most commonly used and most useful ones are. Are all map projections loss-less (one-to-one)? Commented Jan 20, 2023 at 15:03
• Before I saw your comment I added small print that transformations must be invertible. so that we are excluding (e.g.) squares from positive and negative numbers, where roots can be positive or negative. Does that take care of your comment? I can't think of a useful map projection that isn't one-to-one. It's of interest sometimes to superimpose maps (e.g. of different countries) to compare size or shape but when that is done there is point to keeping areas identifiable. Commented Jan 20, 2023 at 15:15
• Thank you, yes. I would make a guess that the most common lossy transformation is taking the absolute value. (And mp3 and other audio/visual compressions which we don't perceive as lossy.) Commented Jan 20, 2023 at 15:21
• Absolutely, so to speak. The context of the thread is positive data values. The small print is endless: e.g. it's clear in this thread that logarithms are a serious candidate, and equally clear that isn't so for variables that can be zero or negative, although modifications can be defined. Commented Jan 20, 2023 at 15:32