# Comparison of Box-Cox transformed data with different lambdas

I'm trying to compare some data and see if there is a significant Pinteraction value between them. The data is highly skewed and thus I would like to use a transformation; a log transformation results in highly non-normal residuals, thus I am looking for a more appropriate transformation, if it exists. I came upon the Box-Cox transformation, and I'm trying to see if it will work. However, for every dataset I have a unique $$\lambda$$, and thus a different equation of the form using the former, because my $$\lambda$$ value was found to be not zero on all occasions.

My question, therefore, is if I can statistically compare two data sets transformed with different lambda values, or if there is a way to find a lambda value which is the maximum likelihood for both data sets. Or if I've made a horrible mistake.

This is how I found my lambda value, just to make sure I did not make a mistake.

Assume data sets Data1 and Data2, where Data1 is the response variable.

library('MASS')

#Initial regression to get regression object
LM <- lm(Data1 ~ Data2)
LM.b <- boxcox(LM)
#x = lambda values, y = likelihood values
lam <- LM.b$$x lik <- LM.b$$y
lam.lik <- cbind(lam,lik)
#Sort by likelihood to get maximum likelihood lambda
lam.lik.sort <- lam.lik[order(-lik),]
LAM <- lam.lik.sort[1,1]

#Perform regression on transformed values
Data1.trans <- ((Data1^LAM) - 1)/LAM
LM.trans <- lm(Data1.trans ~ Data2)
shapiro.test(LM.trans\$residuals)


You didn't tell us how you "want to compare" the different datasets. But, generally the answer is that for most meaningful analysis, you need the same $$\lambda$$ value for all datasets. The reason is that the Box-Cox transformation not only changes the scale of the data, it also changes the unit of measurement. For a related discussion see Skewness transformation for one but not the other variable?.
So, how to find a common value? If the estimated values you have are not too different, simply choose some convenient common value. Mostly we do not use the estimated $$\lambda$$ directly, we choose some meaningful value close to it, like $$\lambda=1/2$$ (squareroot), $$\lambda=1/3$$ (cube root), $$\lambda=-1$$ (inverse) etc. If the estimated values are so different that this do not work, try to estimate a common value by combining the different datasets in a common model. Just use interactions to represent different models as one common model, see for instance Separate Models vs Flags in the same model