Suppose we want to predict $Y$ given the following $X$ observations:

x = c(abs(rnorm(2500, 0.1, 0.25)), abs(rnorm(2500, 0, 0.05)))
y = (x^0.35) + rnorm(length(x), 0, 0.25)
x = c(x, -x)
y = c(y, -y)

Clearly we have an exponential relationship between the predictor $X$ and $Y$. It's more obvious looking at a local regression plot:

regrplot = function(x, y, main.par="", regr.par=T, lowess.par=T, xlab.par=NULL, ylab.par=NULL)
  plot(x, y, cex=0.5, col="red", main=main.par, xlab=xlab.par, ylab=ylab.par, pch=".")
  if( regr.par )
    abline(lm(y ~ x), col="blue", lwd=1)
  if( lowess.par )
    lines(lowess(x, y), lwd=1, col="darkgreen")

enter image description here

This is a replica of an empirical distribution I have to model and predict.

What are the difference in using splines vs. GLM? Which GLM regression better predicts this kind of distributions?

From what I understand splines are better suited to study empirical distributions, because of the flexibility of the polynomials based on a guided number of knots. GLMs seems to be better to study theoretical distributions. Of course, the drawback with splines is overfitting, especially for prediction.


You are confusing a few issues. A spline function is typically used to relax the linearity assumption of one or more predictors, whether in the context of a GLM or a non-GLM. Loess is a local weighted linear regression, not a spline.

When discussing empirical distributions you really need to talk about the conditional distribution of $Y|X$ which may have little to do with the right hand side of the equation I discussed above. Most models assume a parametric distribution for $Y|X$. The empirical alternative is semi-parametric models such as cumulative probability ordinal response models (e.g., proportional odds and prop. hazards models). These do not assume any model for $Y|X$, effectively using the empirical CDF.

  • $\begingroup$ ok, so splines can be part of a GLM. Wouldn't a 4 knot splines OLS be a good choice for my example? $\endgroup$ – Robert Kubrick Sep 12 '14 at 13:58
  • 1
    $\begingroup$ There is no such thing as a "4-knot OLS". There is a "4-knot spline" in any regression model, and there is "OLS regression" with any form of predictors. $\endgroup$ – Aniko Sep 12 '14 at 14:00
  • $\begingroup$ @Aniko I mean using a 4-knot spline and OLS. $\endgroup$ – Robert Kubrick Sep 12 '14 at 14:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.