Using several linear regression

Question

I am currently trying to build an algorithm to predict a continuous output (Y) from a list of predictors (X). My first idea was to use a simple linear regression to see how it performs. Distribution of residual errors is not normal.

I have a lot of data and I was wondering if I can take advantage to this to split my training dataset into different datasets where the relationship between X and Y would behave differently. I then would train different linear regressions that would perform better on these subsets. My question is : does it bring a significant improvement and how to split my dataset optimally.

NB: the reason why I want to stick to simple linear regression is that I want to be able to make predictions very quickly.

Thanks in advance

Answer 1

If the distribution of the residuals are not normal, then you might want to consider other methods since the predictions and confidence intervals are likely to be misleading. Ease-of-computation doesn't seem like a good enough reason.

In terms of clock cycles, it's more expensive to create models than it is to make predictions from them. I would imagine that you'd create the model(s) relatively infrequently (but use them a lot), and so the model creation speed might be decoupled from your process.

To illustrate this, I created five toy models based on Hadley's fueleconomy dataset:

A couple of things stood out to me:

1. the models took much longer to create than they did to make predictions.
2. if speed is important, it's worth looking at the gputools R package. On my workstation, the GPU optimized linear regression (gpuLm) was about 100x faster to create, and 10x faster to predict, than the standard R lm.