I'm due to teach a workshop on statistics in a week or two, and one topic I will cover bothers me. The reason it bothers me, is that the advice I see, both in textbooks and here often on StackExchange, jars with my own intuition. So, possibly, my own intuition needs fixing before I give this workshop.
The question I have regards transformation of non-normal data. It seems common to recommend that non-normal data be transformed prior to using a parametric test (e.g. t-test), in order to meet the normality requirements. First, of course, if the sample is large enough (compared with the skew), we have CLT, so the transformation seems superfluous from the perspective of "satisfying normality assumptions". But the thing that jars with me is that it is possible, for example, to log-transform two different groups and find that their respective means actually swap places (so the higher mean becomes the lower mean etc), which would lead us to exactly the opposite conclusion than we would have prior to transformation.
What is the general advice for managing this trade-off? My own intuition is that you should log-transform where the behaviour of the DV seems like it should be logarithmic, etc, but not if you cannot justify it. Though I do appreciate that a more symmetrical distribution will often lead to a more meaningful mean.
I suppose I'm looking for three things: (i) is my intuition right that log-transforming needs to be done with extreme caution (and with greater regard to the ultimate meaning of the the log-transformed data/means), (ii) do others feel that the advice often given is too conservative in that regard, (iii) do people out there have clear clean guidelines that can be given to a bunch of NON-statisticians as to when it is a reasonable thing to do? Any ideas as to how to communicate the idea, or build an intuition for them would be highly appreciated.