Why would a statistical model overfit if given a huge data set? My current project may require me to build a model to predict the behavior of a certain group of people. the training data set contains only 6 variables (id is only for identification purposes):
id, age, income, gender, job category, monthly spend

in which monthly spend is the response variable. But the training dataset contains approximately 3 million rows, and the dataset (which contains id, age, income, gender, job category but no response variable) to be predicted contains 1 million rows. My question is: is there any potential problems if I throw too many rows (3 million in this case) into a statistical model?I understand the computational expenses is one of the concern, are there any other concerns? Are there any books / papers that fully explain the data set size issue?
 A: What's important is the number of individuals (rows) compared to the number of coefficients you need to estimate for the model you want to fit. Typical rules of thumb suggest about 20 observations per coefficient as a minimum, so you should be able to estimate up to 150,000 coefficients—surely more than adequate for your four predictors.
In fact you have an opportunity, not a problem, in this case: to fit a rather complex model including non-linear relationships of the response to predictors, & interactions between predictors; which may predict the response much better than a simpler one in which the relationships of the response to predictors are assumed to be linear & additive.
A: There are two sorts of problems you might encounter:
1) Computer problems because the data set is too big. These days, a few million rows with 6 columns is just not that big. But, depending on your program, your computer, your amount of RAM and probably other things, it might bog down.
2) Statistical problems. Here, a problem like you discuss will have one "problem" that I know of: Even tiny effects will be highly significant. This is not really a problem with regression, it's a problem with p values. Better to look at effect sizes (regression parameters).
3) Another sort of problem with your model is not due to number of rows, but the nature of the response variable (monthly spend). Although OLS regression does not make any assumptions about the distribution of the response (only about the error), nevertheless, models with money as the dependent variable often have non-normal errors. In addition, it often makes sense, substantively, to take the log of the response. Whether this is so in your case depends on exactly what you are trying to do. 
