How to conduct linear regression with lots of data? Say we have an absolutely huge dataset, and it's too much to put it all into one linear regression model to train. How can we go about using all of this data?
I was thinking that we could break this up into, say 5 different chunks, and then train on each of these 5 chunks, but then I'm not sure how we would go about weighting each of the models.
 A: How much is too much data? There are plenty of libraries in R that allow you to fit a linear regression on millions of observations without a hitch
Take a look at felm in the lfe package, or biglm in the biglm package.
A: I think another and better approach would be sampling. Statistical sampling is a large field of study, but in applied machine learning, there may be three types of sampling that you are likely to use: simple random sampling, systematic sampling, and stratified sampling.
A: The "too much" or 'big' data is going to be relative to what your computer can process, whether that's a local machine or a high-performance computing center (i.e., a super computer).
One alternative way to think about it is,

*

*given the abundance of data, how can I best perform a regression that can answer my questions?
In general, more data leads to better estimates (i.e., smaller variances around parametric estimates, etc.), but sometimes you don't need all the data.
Also, the luxury of having a lot of data means you can get creative in how you design your analyses. Perhaps, identifying small cohorts of interest and performing regression on them, or doing 5-fold cross-validation approach like you alluded to via chunking into 5 groups.
