5
$\begingroup$

I'm attempting to assess the relationship between two quantitative variables, but the DV is highly skewed (and so are the residuals). I work among biologists who tend to favor non-parametric techniques (e.g., Mann Whitney, Kruskal Wallis). In one part of the paper, a Mann Whitney is used to assess group differences. Although I generally do the Box-Cox when doing simple (or multiple) regression, for continuity I decided to rank transform the DV. Alas, it was all noise (i.e., small effect sizes and non-significant p-values). Out of curiosity, I decided to use a Box-Cox transformation. With that, the p-values became significant and the effect sizes increased.

So, with that background, a couple of questions:

  1. Am I to interpret the discrepancy as due to random fluctuations (i.e., the Box-Cox is committing a Type I error)?

  2. Am I to interpret the rank transformation as more conservative? (i.e., the rank transformation is committing a type II error).

  3. (Related to #1 and #2) Does one transformation tend to be more conservative than the other? Or does it depend on the dataset?

Improve this question
$\endgroup$
5
  • 3
    $\begingroup$ Using rank as a response variable in regression seems a strange thing to do. It's likely to pull in outliers but at the same time it is highly liable to stretch apart values that are very close. Also, it is hard to compare different studies. With a transformation from Box-Cox family, at least you have something that is invertible and can be related to a response scale that you care about scientifically and/or practically. Regardless of that, you can plot rank, transformed values and original responses for your data and think about whether they do what you want them to do. $\endgroup$
    – Nick Cox
    Commented May 8, 2015 at 15:08
  • $\begingroup$ Good point, @NickCox. I didn't think about the "invertibility" of the box-cox family. $\endgroup$
    – dfife
    Commented May 8, 2015 at 15:18
  • 1
    $\begingroup$ Ranks are invertible if interpolation is allowed; my point is rather that that is an idiosyncratic back-transformation. $\endgroup$
    – Nick Cox
    Commented May 8, 2015 at 15:20
  • 2
    $\begingroup$ @Nick gives many good and interesting reasons not to use ranks as a dependent variable. But one that ought to trump them all is that their error structure strongly violates assumptions of linear models: the variation is heteroscedastic and intercorrelated. Consequently any p-values (computed as if those assumptions were true) are practically meaningless. $\endgroup$
    – whuber ♦
    Commented May 8, 2015 at 15:50
  • 3
    $\begingroup$ @whuber Implausible or absurd assumptions about error structure are indeed compelling extra reasons for not doing this. But I'd aver that the ranks not being a useful scale for modelling response trumps any problems with error structure. $\endgroup$
    – Nick Cox
    Commented May 8, 2015 at 17:20

1 Answer 1

Reset to default
4
$\begingroup$

Given that your area tends to favor non-parametric tests, my answer will assume that the ordinal properties of your dependent variable are more important than the interval properties, and so back-transformation is less important to you than is determining whether or not some monotonic relationship exists (please correct if I'm wrong).

Box-Cox transformation could increase Type I error rates if the free parameter $\lambda$ is selected by maximizing $R^2$, or by some equivalent procedure such as maximizing log-likelihood. For example, the boxcox() function in R does this by default, and so that could be a concern. This would be less of a concern if $\lambda$ were chosen solely on the basis of normalizing the dependent variable (without consideration of the IVs), but of course, such a strategy might also lead to less normal residuals.

Considering that the question is about regression, it seems your goal is to get the residuals approximately normally distributed. For that reason, the answer to all of your questions is "it depends on the dataset." Although Box-Cox is quite flexible, it won't always be able to address non-normality, and it may sometimes be less useful than a simple rank transformation. As an extreme example, a Box-Cox transformation would do little to normalize a sample from a Cauchy distribution because of its symmetry and heavy tails, but a rank transformation would at least pull in both tails. I am not aware of a comparison between Box-Cox and rank-transformation in regression, but you can get some sense by looking at a comparison in correlation. In the case of correlation, the rank transformation (with the Spearman $r_s$) often leads to smaller Type I and II error rates for symmetric non-normality, but the Box-Cox is sometimes superior for asymmetric, skewed distributions (Bishara & Hittner, 2012).

References:

Bishara, A. J., & Hittner, J. B. (2012). Testing the significance of a correlation with nonnormal data: comparison of Pearson, Spearman, transformation, and resampling approaches. Psychological methods, 17(3), 399.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ I don't quite understand the comparisons you are making because the "rank transformation" is not a re-expression of the values in the way a Box-Cox transformation is. The difference is that the ranks depend on the dataset and are undefined for any other values, whereas the Box-Cox transformation (although estimated from the data) does not depend on the dataset and can be applied to any other values. This makes it difficult to see how ranks could be of use in problems of regression (as opposed to correlation or association). This question appears to be from a regression perspective. $\endgroup$
    – whuber ♦
    Commented May 14, 2015 at 15:49
  • 2
    $\begingroup$ I agree that ranks pose problems for out of sample prediction, although rank transformed regression isn't as horrible as one might expect it to be on the face of it (Iman & Conover, 1979). I am interpreting @dfife's question, which focuses on nonparametric tests and p-values, as an interest is in establishing the existence of a monotonic relationship rather than applying the regression coefficients to other datasets. For this specific goal, I am suggesting that the limitations of rank transformation may sometimes be outweighed by its benefits. $\endgroup$
    – Anthony
    Commented May 14, 2015 at 16:11
  • 1
    $\begingroup$ the box-cox transform parameter as usually estimated (by maximizing a likelihood) is more trying to stabilize variance that to normalize residuals! And frankly, constant variance is a much more important assumption than normality. More of a concern with box-cox is that the transform changes the meaning of the parameters in a linear model! so, if that is a concern (it should be), then some glm might be better. $\endgroup$
    – kjetil b halvorsen ♦
    Commented Dec 12, 2016 at 5:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.