Comparison of Box-Cox transformed data with different lambdas

Question

I'm trying to compare some data and see if there is a significant P_interaction value between them. The data is highly skewed and thus I would like to use a transformation; a log transformation results in highly non-normal residuals, thus I am looking for a more appropriate transformation, if it exists. I came upon the Box-Cox transformation, and I'm trying to see if it will work. However, for every dataset I have a unique $\lambda$, and thus a different equation of the form

using the former, because my $\lambda$ value was found to be not zero on all occasions.

My question, therefore, is if I can statistically compare two data sets transformed with different lambda values, or if there is a way to find a lambda value which is the maximum likelihood for both data sets. Or if I've made a horrible mistake.

---

This is how I found my lambda value, just to make sure I did not make a mistake.

Assume data sets Data1 and Data2, where Data1 is the response variable.

library('MASS')

    #Initial regression to get regression object
LM <- lm(Data1 ~ Data2)
LM.b <- boxcox(LM)
    #x = lambda values, y = likelihood values
lam <- LM.b$x
lik <- LM.b$y
lam.lik <- cbind(lam,lik)
    #Sort by likelihood to get maximum likelihood lambda 
lam.lik.sort <- lam.lik[order(-lik),]
LAM <- lam.lik.sort[1,1]

    #Perform regression on transformed values
Data1.trans <- ((Data1^LAM) - 1)/LAM
LM.trans <- lm(Data1.trans ~ Data2)
shapiro.test(LM.trans$residuals)   

Answer 1

You didn't tell us how you "want to compare" the different datasets. But, generally the answer is that for most meaningful analysis, you need the same $\lambda$ value for all datasets. The reason is that the Box-Cox transformation not only changes the scale of the data, it also changes the unit of measurement. For a related discussion see Skewness transformation for one but not the other variable?.

So, how to find a common value? If the estimated values you have are not too different, simply choose some convenient common value. Mostly we do not use the estimated $\lambda$ directly, we choose some meaningful value close to it, like $\lambda=1/2$ (squareroot), $\lambda=1/3$ (cube root), $\lambda=-1$ (inverse) etc. If the estimated values are so different that this do not work, try to estimate a common value by combining the different datasets in a common model. Just use interactions to represent different models as one common model, see for instance Separate Models vs Flags in the same model