I'm using SciPy's boxcox transformation on a continuous variable (>0).

The lambda parameter is None, so the function finds the lambda that maximizes the log-likelihood function and returns it as the second output argument.

The SciPy's documentation defines the transformation as:

   y = (x**lambda - 1) / lambda,  for lambda > 0
   log(x),                      for lambda = 0

The problem is that in some realizations the transformation returns a NEGATIVE (close to zero) lambda value.

1) From what I understand given the definition this is not the expected behavior, is it?

2) Which part of the transformation should I apply to invert the result?


There is on the face of it nothing unexpected here at all.

I am not familiar with this implementation and we cannot see your data, but the principles are generic. A value of lambda of $0$ would imply a logarithmic transformation and a value of $-1$ would imply a reciprocal transformation. Values in between would imply negative powers in between. All of these are are defined, indeed standard, transformations for values that are all positive, as is explicit here.

The way in which Box-Cox transformations are used varies, but I recommend following the style in the original Box and Cox paper, as treating lambda estimates as pointing towards one of a small number of simply defined transformations. Tastes, styles and experiences differ, but most problems seem to call for one of this short list of candidates: reciprocal, logarithmic, cube root or square root transformations.

Thus if lambda emerged as $-0.1$ or $0.1$, I would use logarithms, not a power $-0.1$ or $0.1$.

The two detailed examples in the original Box and Cox paper pointed respectively to logarithmic and reciprocal transformations.

Disclaimer: Sir David Cox and I are not related.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.