6
$\begingroup$

Suppose, we have a set of measurements of some quantity in some units of measurement. We also have a nice model that heavily relies on the properties of the Gaussian distribution. The model is tailored for data in some specific units of measurement with some physical meaning behind (like watt, ohm, etc.). It turns out that the distribution of the data does not exactly follow the normal distribution and has some undesired features (like skewness). We apply the popular Box-Cox transformation and obtain a more or less normally distributed data set. The problem now is that we have logarithms, powers, etc. of the original measurements, which contradicts with our nice model.

The question is, what can one do in such a situation? I need to change the model such that it can handle the new data? And in general, if I got everything correctly, why do people what to study transformed data that have lost their physical meaning? Because, at the end of the day, one will, probably, have to return back to the original units of measurement.

$\endgroup$
6
  • 6
    $\begingroup$ Are you, for example, claiming that an ohm has a privileged physical meaning that, say, 1/ohm does not have? (This is not true: 1/ohm has a physical meaning too, in terms of conductance instead of resistance.) Often, a well-chosen Box-Cox parameter reveals a physical meaning that was not apparent in the original formulation. $\endgroup$
    – whuber
    Commented Mar 21, 2012 at 16:24
  • 1
    $\begingroup$ If the model relies on physics and is meaningless after transformations, then you have to remove the normality assumption. $\endgroup$
    – Aniko
    Commented Mar 21, 2012 at 18:30
  • $\begingroup$ @whuber, yes, I absolutely agree with you, of course, 1/ohm does have its own physical meaning. What I was trying to say is that, for instance, if you are interested in the human height and you measure it directly, what can logarithm or some power of your measurements do good for you? You have substituted the subject of your research with something completely different. $\endgroup$
    – Ivan
    Commented Mar 21, 2012 at 21:07
  • $\begingroup$ @Aniko, yes, probably, the assumption about the normality of the date is incorrect in such a situation, and one should start looking into some other direction of solving the problem. I was actually trying to figure out whether there is a way of improving, justifying the existing model without changing it dramatically, with the only transformation of the input. $\endgroup$
    – Ivan
    Commented Mar 21, 2012 at 21:15
  • 3
    $\begingroup$ The cube of the height will be closely related to weight; the square of the height, to skin surface area (and thence to risk associated with dermal contact to contaminants); etc. Thus, it is not automatic that a Box-Cox parameter will be without physical meaning or completely lack interpretability. Having said that, evidently it is wise--but not necessary--to limit choices of the parameter to values that might be amenable to interpretation. $\endgroup$
    – whuber
    Commented Mar 22, 2012 at 2:13

2 Answers 2

1
$\begingroup$

First of all, if you mean a linear regression model, it does not assume the data are normally distributed, it assumes the error as estimated by the residuals is normally distributed (in fact, they should be iid $\mathcal{N}(0,\sigma)$).

Second, if that assumption is violated and you want to keep your original units, you can use some other form of regression - there are a variety of robust regression models, loess models, spline models, etc.

$\endgroup$
2
  • $\begingroup$ I am sorry, I should have explained the context better. By model I did not mean a linear regression model, I meant a mathematical formulation of a physical system. In other words, there is an equation $f(a, b, c, d)$ describing how the system evolves and one of the arguments is assumed to be uncertain, let say $a \sim \mathcal{N}(\mu, \sigma^2)$, so we have its nominal value, $\mu$, and some deviation, $\sigma$, from it. $\endgroup$
    – Ivan
    Commented Mar 21, 2012 at 12:01
  • 1
    $\begingroup$ @Peter, if the errors are not Normal the cures you suggest may not help, or may not be required. E.g. if the mean is specified correctly, and the errors have constant variance, Normality is a non-issue for inference on parameters in the mean. (See e.g. McCullagh and Nelder) Fitting a spline representation of a covariate helps the mean be "less wrong", but does nothing about non-constant variance. "Robust regression" (not use of robust standard errors) provides robustness to erroneous data, but can reduce the influence of the most informative good data points. $\endgroup$
    – guest
    Commented Mar 22, 2012 at 5:54
1
$\begingroup$

It sounds like your model is of this form; $$Y_i|x_i = f(x_i, \beta) + \epsilon_i,$$ where $Y_i$ denotes the $i$th measured outcome, $x_i$ is a vector of covariates for that outcome (i.e. experimental circumstances), which with (unknown) parameters $\beta$ determines the expected value $f(x_i, \beta)$ for that observation. The $\epsilon_i$ are the error terms, which describe everything that affects $Y_i$ not captured by $f(x_i, \beta)$ - i.e. experimental errors.

Before getting into analysis, it's always good to ask "why do you want to do this analysis?". The answer to this question determines how much you should worry about Normality, or whether a transformation is needed. Suppose, as is common, you want inference on the value of $\beta$. If you believe that $f(x_i, \beta)$ captures the mean value of $Y_i$ correctly, and you believe that $Var(\epsilon_i)$ is the same for every measurement, then classical linear regression can be used for inference about the value of $\beta$. Despite what many textbooks advise, you do not need Normality here; in reasonable sample sizes your confidence intervals and tests will be almost perfectly-accurately calibrated.

If you still want inference, but don't believe the constant variance, use robust standard error estimates. If you don't believe the mean follows $f(x_i, \beta)$ or that the variance is constant, robust standard error estimates still give you accurate inference on the best-fitting line of the form $f(x_i, \beta)$, where "best-fitting" means "least-squares". And if you don't believe the mean follows $f(x_i, \beta)$, or that the best-fitting line of this form is a useful thing to know, you can always fit a more flexible mean - spline representations of covariates $x_i$ are a good way to do this. Absolutely none of the methods listed require Normality - or transformations of the $Y_i$.

So when do we require Normality? If you want to do predictions, of new $Y_i$, for most methods you'll need a model (though it need not assume Normality). If you want to compare models, well, you'll need some models, but that's a tautology. If you have a tiny sample size, doing model-based inference on $\beta$ may be the only viable approach - but then you'd likely have no way of assessing whether your assumption of Normality (or whatever you assumed) was reasonable.

When do we need Box-Cox? If we have little idea about the form of $f(x_i, \beta)$, but believe that errors around $f(x_i, \beta)$ "should" be Normal, then Box-Cox may help find a better form for $f(x_i, \beta)$. But it relies on there being underlying Normality, at the "right" model, and this is hard to justify in many situations.

In short, rather than deal with hard-to-justify transformations, there is a lot you can do with just a mean model. If the original units of measurement help you (and your colleagues) think about what the data tells them, I recommend hanging on to those units, if possible.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.