4
$\begingroup$

I've seen in some of the books of regression analysis or statistics it is recommended that for non-linear associations, it is much better to apply linear regression using re-expressing the data than fitting a suitable curve to them. However, no convincing reason is given to avoid applying curve fitting in place of re-expressing; for example, it is frequently stated that the procedure of curve fitting is much more complicated that that of the re-expression, but these days using powerful computers available everything can be done and it's not a big deal at all.

I'm curious to know that what are the other main drawbacks of curve fitting? Moreover, is always re-expressing better than curve fitting?

$\endgroup$
5
  • 1
    $\begingroup$ Can you give an example (title & author of a book or other document that says this, preferably with a direct quote and page number)? It's easier to respond to something when you can see what was actually said. $\:$ [I would often - but not always - give the opposite advice. A nonparametric or semiparametric fit is reasonably easy to estimate and provide information from, but suitable inference from a transformed model involves several potentially tricky steps -- not least, actually finding a transformation that works] $\endgroup$
    – Glen_b
    Jul 4, 2016 at 7:03
  • $\begingroup$ @Glen_b, thank you for the help. For example, in the book titled " Stats Data and Models" it is stated that re-expressing is better, page 248. The authors state that in addition to the simplicity, linear models are more understandable and getting data straightened gives us the opportunity to use many statistical concepts. $\endgroup$
    – Federico
    Jul 4, 2016 at 7:19
  • $\begingroup$ De Veaux Velleman and Bock? Which edition are you looking at? $\endgroup$
    – Glen_b
    Jul 4, 2016 at 8:02
  • $\begingroup$ Yes. Third edition. $\endgroup$
    – Federico
    Jul 4, 2016 at 8:40
  • $\begingroup$ In the absence of a direct quote and context I'll see if I can get hold of a copy. I may be able to get a look at 2ed in a few days $\endgroup$
    – Glen_b
    Jul 4, 2016 at 8:50

1 Answer 1

3
$\begingroup$

I couldn't get the edition of De Veaux, Velleman & Bock (Stats: Data and Models) you mentioned, only an older version, so the page numbers won't correspond but I assume we're mainly dealing with the chapter on Re-expressing Data.

Oddly enough, 25 years ago I'd probably have mostly agreed with their stance; now, however, I'll restrict my agreement to qualified agreement within a more limited scope

In the context of what they discuss, much of the advice is okay, but it hardly presents a complete or unbiased picture. (However, it's understandable they might take the stance they do since the complete picture would involve a lot more knowledge/background than they have to work with in an elementary book, and would require introducing methods beyond the scope of the book.)

They give three goals of transformation - symmetry, equal variance and linearity (approximate in each case)

If you want to use ordinary regression as-is, for inferential purposes (hypothesis testing, CIs and so on), these are all considerations in one way or another (linearity is most important), but there are a number of alternatives that don't make these assumptions.

  • you don't have to assume normality to perform inference using least squares (one might use a permutation test to test whether a coefficient is different from zero, for example, or use a bootstrap confidence interval)

  • you might make a different parametric assumption but remain within least -squares; if it's not amenable to algebraic calculation you can always use simulation to work out the situation.

  • you don't need to assume constant variance; with a regression framework there's ways of adjusting standard errors of parameters for heteroskedasticity, for example, or you can make a different assumption about heteroskedasticity, or you can model spread as well as mean.

  • you don't need to use least squares; there are many other ways of fitting models. With GLMs for example, they're only least squares at the normal.

  • you don't need linearity -- there are parametric (e.g. GLMs, nonlinear least squares) and nonparametric alternatives (local linear regression or spline models for example)

Those three goals that they present are rarely compatible in practice -- e.g. in the exponential family; consider the Poisson, where the asymptotic symmetrizing transform is the 2/3 power, the asymptotic variance stabilizing transform is the square root, but the natural link (a transform often used to approximately linearize relationships in this instance) is the log (and you can't take log of zero counts; it's not generally used to transform the data but in a slightly different way), or the gamma (with shape held constant), where the asymptotic symmetrizing transformation is cube root, while the corresponding variance stabilizing transformation is log.

More generally,

Variance stabilizing $v(u)= \int^u \frac{C}{V(\mu)^\frac12} d\mu$

Symmetrizing $s(u)= \int^u \frac{C}{V(\mu)^\frac13} d\mu$ -- for exponential family

So for that widely used exponential-family (and indeed, usually more broadly) these will be different unless V is constant (which it is for the Gaussian case)

In the section of De Veaux et al that I looked at, the relationships discussed are

  • mostly between physical quantities or monetary amounts

  • monotonic

and the transformations considered are all (Tukey) "power-ladder" transforms

So:

For physical measurements, often the transformations likely to straighten relationships are obvious (such as inverting quantities that are ratios to get the ratio the "right way around" for the variable you're relating it to). Indeed, considerations of physical laws or even just of the units will often lead directly to a transformation on the power ladder

For money related variables a number of considerations, from scale issues (conclusions shouldn't change if you move from dollars to cents, say) to the way things like interest and inflation operates - multiplicatively - all tend to lead one to consider log-transforms.

Even when not dealing with physical measurements or money, given monotonic relationships, often a power transformation of X or Y will tend to lead to straighter relationships

So given that context, yes, often transformation may be reasonably simple

Some things to ponder:

a) What do you do when you can only reasonably satisfy one of those goals? (sometimes you may be able to do well on all of them at once, but often you cannot)

b) What about when the quantities are not physical measurements so the form of transformations you might consider are less obvious? What about when relationships are not monotonic?

c) How do you interpret for a lay audience a relationship between say ${Y}^\frac13$ and $1/X_1^\frac12$ and $\log(X_2)$? People are not used to thinking in terms of relationships between cube-roots and inverse square roots.

d) An expected value (/forecast mean) for a transformed relationship doesn't back-transform. i.e. $g^{-1}(E[g(X)])\neq E(X)$. So if you want a fitted value or a mean prediction on the original scale you can't just invert the transformation.

e) we would need to account for the search for suitable parameters. e.g. if we could consider transforming X or Y or both, even if we only considered power transformations we're effectively optimizing our model over two additional parameters; if we don't account in some way for the impact of that model-choice/optimization our models will look much better than they are in sample, our other parameters will tend to be biased (away from zero), our standard errors will be too small, and so on.


By comparison, one can model nonlinear relationships via nonlinear least squares, or we might model nonlinear relationships and heteroskedasticity on the original scale (e.g. via GLMs or GAMs, - where symmetry is also not assumed). Or one can look at various forms of nonparametric regression (nonparametric relationships between Y and X, rather than nonparametric distributional assumptions) and get expected values on the original scale, which conclusions may be easier to explain.

So I would say it depends on what you're modelling, and why, on who your audience is, what tools are available, and what kinds of things you want to say about the relationship.

Transformation can be useful, but it's often not the best choice available, and it's not all as simple as that chapter suggests.

$\endgroup$
1
  • $\begingroup$ Thank you so much for your great explanation. I can now understand the subject far better. $\endgroup$
    – Federico
    Jul 7, 2016 at 5:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.