Using several linear regression

I am currently trying to build an algorithm to predict a continuous output (Y) from a list of predictors (X). My first idea was to use a simple linear regression to see how it performs. Distribution of residual errors is not normal.

I have a lot of data and I was wondering if I can take advantage to this to split my training dataset into different datasets where the relationship between X and Y would behave differently. I then would train different linear regressions that would perform better on these subsets. My question is : does it bring a significant improvement and how to split my dataset optimally.

NB: the reason why I want to stick to simple linear regression is that I want to be able to make predictions very quickly.

2. if speed is important, it's worth looking at the gputools R package. On my workstation, the GPU optimized linear regression (gpuLm) was about 100x faster to create, and 10x faster to predict, than the standard R lm.