16
$\begingroup$

In Andy Field's Discovering Statistics Using SPSS he states that all variables have to be transformed.

However in the publication: "Examining spatially varying relationships between land use and water quality using geographically weighted regression I: Model design and evaluation" they specifically state that only the non-normal variables were transformed.

Is this analysis specific? For instance, in a comparison of means, comparing logs to raw data would obviously yield a significant difference, whereas when using something like regression to investigate the relationship between variables it becomes less important.

Edit: Here is the full text page in the "Data Transformation" section:

And here is the link to the paper: http://www.sciencedirect.com/science/article/pii/S0048969708009121

$\endgroup$
1
  • 21
    $\begingroup$ Based on the new image you posted, the author appears to confuse "variable" with "observation." On the top of p. 154 he rightly emphasizes that when you transform a variable, you must transform all its values (observations) in the same way, for otherwise it becomes impossible to compare anything. (Asserting this "won't change the relationships between variables" requires an extremely generous interpretation to be correct, though.) The bottom of p. 154 is so plainly wrong, sentence by sentence (even into the footnotes), that it's not even worth the time to enumerate all the problems with it. $\endgroup$
    – whuber
    Commented Jun 16, 2015 at 15:52

2 Answers 2

19
$\begingroup$

You quote several pieces of advice, all of which is no doubt intended helpfully, but it is difficult to find much merit in any of it.

In each case I rely totally on what you cite as a summary. In the authors' defence I would like to believe that they add appropriate qualifications in surrounding or other material. (Full bibliographic references in usual name(s), date, title, (publisher, place) or (journal title, volume, pages) format would enhance the question.)

Field

This advice is intended helpfully, but is at best vastly oversimplified. Field's advice seems to be intended generally; for example, the reference to Levene's test implies some temporary focus on analysis of variance.

For example, suppose I have one predictor that on various grounds should be logged and another indicator variable that is $(1,0)$. The latter (a) cannot be logged (b) should not be logged. (Indeed any transformation of an indicator variable to any two distinct values has no important effect.)

More generally, it is common -- in many fields the usual situation -- that some predictors should be transformed and the rest left as is.

It's true that encountering in a paper or dissertation a mixture of transformations applied differently to different predictors (including as a special case, identity transformation, or leaving as is) is often a matter of concern for a reader. Is the mix a well thought out set of choices, or was it arbitrary and capricious?

Furthermore, in a series of studies consistency of approach (always applying logarithms to a response, or never doing it) does aid enormously in comparing results, and differing approach makes it more difficult.

But that's not to say there could never be reasons for a mix of transformations.

I don't see that most of the section you cite has much bearing on the key advice you highlight in yellow. This in itself is a matter of concern: it's a strange business to announce an absolute rule and then not really to explain it. Conversely, the injunction "Remember" suggests that Field's grounds were supplied earlier in the book.

Anonymous paper

The context here is regression models. As often, talking of OLS strangely emphasises estimation method rather than model, but we can understand what is intended. GWR I construe as geographically weighted regression.

The argument here is that you should transform non-normal predictors and leave the others as is. Again, this raises a question about what you can and should do with indicator variables, which cannot be normally distributed (which as above can be answered by pointing out that non-normality in that case is not a problem). But the injunction has it backwards in implying that it's non-normality of predictors that is the problem. Not so; it's no part of regression modelling to assume anything about marginal distributions of the predictors.

In practice, if you make predictors more nearly normal, then you will often be applying transformations that make the functional form $X\beta$ more nearly right for the data, which I would assert to be the major reason for transformation, despite the enormous emphasis on error structure in many texts. In other words, logging predictors to get them closer to normality can be doing the right thing for the wrong reason if you get closer to linearity in the transformed space.

There is so much extraordinarily good advice on transformations in this forum that I have focused on discussing what you cite.

P.S. You add a statement starting "For instance, in a comparison of means, comparing logs to raw data would obviously yield a significant difference." I am not clear what you have in mind, but comparing values for one group with logarithms of values for another group would just be nonsensical. I don't understand the rest of your statement at all.

$\endgroup$
6
  • $\begingroup$ Nick, I wanted to get my point across quickly and concisely, which I feel I did. In the world of Google, I have provided enough information to easily access the original documents, should they be required. Thank you for answering, albeit in a round about way, you provided me with exactly the information I was seeking: Having to transform all variables, as is the suggestion by Field, in his Data Transformation Section is an incorrect approach to data transformation. $\endgroup$ Commented Jun 16, 2015 at 15:30
  • 17
    $\begingroup$ +1. I marvel at how tactfully you have managed to address material that is so outright wrong. Skimming pages here and there in the SPSS book provides insight into some of the really confused questions we get on this site: I think they must come from readers of that book. It is full of errors, misinformation, and outright confabulation. $\endgroup$
    – whuber
    Commented Jun 16, 2015 at 15:37
  • 1
    $\begingroup$ @I Heart Beats Pleased you found the answer helpful, but my request for proper referencing stands. You might as well say that incomplete references are always defensible because interested people can always Google. On the contrary, good scholarship and science are helped by good bibliographic practice, giving full details and not making (many) readers do unnecessary work. $\endgroup$
    – Nick Cox
    Commented Jun 16, 2015 at 15:42
  • 9
    $\begingroup$ Thanks for enhancing the references. You have quoted more from Field. There is an extra section visible including the assertion that "transforming the data won't change the relationships between variables". Either that pivots on an idiosyncratic meaning of "relationship", or (more likely, I fear) it is unhelpful, indeed quite wrong. I regret (in one sense) to agree with @whuber about the book in question on the evidence in front of us. (Update: whuber was making essentially the same point simultaneously: see his comment on the question.) $\endgroup$
    – Nick Cox
    Commented Jun 16, 2015 at 15:55
  • 1
    $\begingroup$ I got to see a copy of the book, & what precedes this page concerns only comparisons between groups; the subsequent chapter on regression makes no mention of transforming predictors (except in the context of the Box-Tidwell test). Putting that together with the example in the 2nd para. of p.154, I think it's clear the author intends only to warn against inappropriate comparisons between quantities on different measurement scales (IMHO he makes rather a hash of it). $\endgroup$ Commented Feb 25, 2022 at 10:21
11
$\begingroup$

First of all, both quotes are misleading insofar as any transformation applied to data intended for use in a regression model is not done to make the variable PDFs more normally distributed, it's done to make the model residuals more symmetric since one assumption in classic regression is that the errors are Gaussian. This implies a deeper level of rigor and stringency than merely symmetrizing a PDF.

Moreover both quotes are weak in that neither one delves into the motivations for their prescriptions (at least based on the information provided). As it happens, I disagree with both.

In the passage you've highlighted, the SPSS book claims that mixtures of transformations (e.g., natural log for one variable, sq root for another) are not permitted. Why is this illegal? Mixtures of transformations violate no regression assumptions that I'm aware of. Please check any regression texts on regression assumptions to confirm that this is the case. Transformation mixtures might present a substantive descriptive problem in terms of their interpretation, but that's not a question of whether or not mixtures are illegal. The SPSS guy is wrong.

As far as the second text goes, again, transformations are totally a matter of analyst choice -- whether one does them at all, transforms all inputs or some variables and not others. None of this violates any assumptions.

Where I think the second quote goes off the rails is in the assertion that, "...to avoid the potential multicollinearity...only one land use indicator (was used)..." This is blatantly bad advice and sounds like the kind of thing some analysts will do as a dimension reduction technique where they will factor analyze a bunch of variables and pick the highest loading variable on each factor. This heuristic has been around for years and is not one I either use or recommend. Again, this a matter of analyst preference and training. But this point is not targeted to answering your specific questions.

At the end of the day, both quotes come off as being assertions of the authors' opinions in the absence of any supporting evidence, based on the information provided.

$\endgroup$
2
  • 9
    $\begingroup$ We make broadly similar points, but I want to add that good texts explain that Gaussian errors are the least important assumption in regression modelling and not needed for many purposes. $\endgroup$
    – Nick Cox
    Commented Jun 16, 2015 at 15:46
  • 4
    $\begingroup$ Good text: stat.columbia.edu/~gelman/arm : ) $\endgroup$ Commented Jun 16, 2015 at 20:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.