2
$\begingroup$

I understand that there are various assumptions for statistical techniques such as regression. A common assumption is that of data being normally distributed. In cases where data is not normally distributed, it can often be transformed (normalized?) through, for example, taking the log of every case of the variable.

I can see that this, and other forms of transformation, (often) makes the distribution normal, which is a condition for regression. However, have we not manipulated the variable? How can it be that inferences made based on this manipulated variable still are valid?

$\endgroup$
  • 5
    $\begingroup$ The fact that you can refer to a tag shows that similar questions on transformations have often been asked before. Are you sure that none of them address your question? As it happens, the emphasis here is pretty much the wrong way round, as normality of marginal or even conditional distribution is just about the least important assumption which might be more nearly satisfied after a transformation: approximate additivity, linearity and roughly constant variances are all more important. $\endgroup$ – Nick Cox Apr 20 '14 at 17:06
  • $\begingroup$ We have manipulated the variable, yes. More than just the distribution is affected. But "manipulating a variable" is not of itself automatically a problem, if all the assumptions are satisfied, and assuming we can transform the information we need back. $\endgroup$ – Glen_b Apr 20 '14 at 22:25
  • 1
    $\begingroup$ One is tempted to turn the question around: what assurance have you that the data, as originally given to you, are represented in a way that is meaningful for your analysis? How can it be that inferences based on a (thoughtlessly!) untransformed variable are valid in the first place? $\endgroup$ – whuber Nov 27 '18 at 23:23
2
$\begingroup$

Taking the logarithmic transform as an example, this is an example of an invertible transformation, meaning that you can transform back the other way to recover the original variable. If you have an original random variable $X$ and you define the new variable $Y \equiv \ln X$ and then model this latter variable (e.g., by a model that uses a normal distribution) then you can also use the transformation $X = \exp (Y)$ to recover you original variable. This means that if you are modelling using $Y$, you should be able to translate your predictions about $Y$ to predictions inferences about your original variable $X$. Just to give a simple example, you could translate a prediction interval for $Y$ back to being a prediction interval for $X$:

$$\begin{matrix} \text{Prediction interval for }Y & & & [L \leqslant Y \leqslant U], \\[6pt] \text{Prediction interval for }X & & & \quad \quad \quad \quad \text{ } \text{ } \text{ } [\exp(L) \leqslant X \leqslant \exp(U)]. \\[6pt] \end{matrix}$$

When dealing with models that use transformations, so long as these transformations are invertible (e.g., the logarithmic transformation), it is always possible ---in principle--- to recover predictions about the original untransformed variable. One must take some care in the translation of predictions to ensure that appropriate correspondence occurs. In particular, unbiased point estimates of a transformed variable will generally correspond to biased estimates for the untransformed variable if a nonlinear transformation is used.

$\endgroup$
  • $\begingroup$ I think aspects of this account might be misleading insofar as they refer to confidence intervals "for [random variables] $X$," whereas a CI applies to a parameter and the parameter usually does not remain the same upon transformation. This is more fundamental than the distinction between biased and unbiased estimates. $\endgroup$ – whuber Nov 27 '18 at 23:43
  • 1
    $\begingroup$ I guess I was actually thinking of prediction intervals. I'll amend. $\endgroup$ – Ben Nov 28 '18 at 0:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.