I am currently trying to build an algorithm to predict a continuous output (Y) from a list of predictors (X). My first idea was to use a simple linear regression to see how it performs. Distribution of residual errors is not normal.

I have a lot of data and I was wondering if I can take advantage to this to split my training dataset into different datasets where the relationship between X and Y would behave differently. I then would train different linear regressions that would perform better on these subsets. My question is : does it bring a significant improvement and how to split my dataset optimally.

NB: the reason why I want to stick to simple linear regression is that I want to be able to make predictions very quickly.

Thanks in advance


1 Answer 1


If the distribution of the residuals are not normal, then you might want to consider other methods since the predictions and confidence intervals are likely to be misleading. Ease-of-computation doesn't seem like a good enough reason.

In terms of clock cycles, it's more expensive to create models than it is to make predictions from them. I would imagine that you'd create the model(s) relatively infrequently (but use them a lot), and so the model creation speed might be decoupled from your process.

To illustrate this, I created five toy models based on Hadley's fueleconomy dataset:

model creation vs model prediction timings

A couple of things stood out to me:

  1. the models took much longer to create than they did to make predictions.
  2. if speed is important, it's worth looking at the gputools R package. On my workstation, the GPU optimized linear regression (gpuLm) was about 100x faster to create, and 10x faster to predict, than the standard R lm.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.