Do you need to split data for Linear Regression? So as far as I understand, the purpose of splitting data into training data and test data is to ensure that you're not overfitting the data. Therefore, why is it necessary to split data into training and testing data in a linear regression? Is there a risk of overfitting data in linear regression? Is there something I'm missing?
 A: Statistics community and machine learning community have "different" ways to control over-fitting. 


*

*Many statisticians follow "The Principle of Parsimony (Occam’s Razor)", which means "given a set of equally good explanations
for a given phenomenon, the correct explanation is the simplest explanation". As a results, people adds variables to the model carefully (run many hypothesis testing before adding it). There are also many regression diagnostic tools to check the validity of the model. In such setting, even without testing data set, people can avoid over fitting effectively. Note that, you will not often see linear models from statistics community with thousand variables.

*For people in machine learning community, they check assumptions less comparing to statisticians, and you may see people use linear model on thousands or millions of variables often. They way of controlling over fitting is using the testing data.
In sum, if it can fit, it can over-fit. Although linear model have high bias, it is also possible to have over fitting problem. People in different community have different ways to control the over fitting problem. And using a testing data set is one of the way. If the features / independent variables are large, and added into the model without careful validation, then testing data is needed.
A: In this context, the term overfitting is usual used to express that a model performs well on the training set but fails to accomplish good results on additional data (= test data).
Least squares using all data will perfom reasonable well on the underlying data (in terms of predicition error) as the least square model is fitted with the use of exactly these data points.
But this, of course, does not guarantee that it will perfom identical good on any new data - especially in practice with real data sets. Of course this holds for other statistical methods as well. 
If one has the possibility to divide the dataset into a training and a test set, one should do so in order to measure its performance.
A: Linear regression model can overfit to your training data. This is the function that is learned:
$y = w_1x_1+w_2x_2+\ldots+w_nx_n$
When you have many variables without enough data, it is possible that your model overfits to data by overweighting unimportant variables. 
Just as a remark: You split data into training and test sets to be able to obtain a realistic evaluation of your learned model. If you evaluate your learned model with the training data, you obtain an optimistic measure of the goodness of your model. So, you should use a separate set (a set that is not seen during training) to obtain a realistic evaluation of your model. 
A: If you are building a model for prediction, it is necessary to split the data to avoid the over-fitting. If the goal of linear regression is just to study and analyze the data then it is not required to split the data.
A: Actually there is no need to split the data , you can fit your regression to the entire data. probably they are splitting the data to use test data to predict the values. More data will calculate the coefficients more accurately and hence the prediction.
