I'm comparing the LR and NB performance on different datasets. And suddenly I am wondering what if we have a big dataset that is infinitely large (at least ensure both models trained to its best??). How would two algorithms work? Usually, LR tends to be overfitting and it requires regularisation. Would in this case it also tend to be overfitting?
Usually, LR tends to be overfitting and it requires regularisation. Would in this case it also tend to be overfitting?
I would not agree. Linear regression is a linear model, comparing with popular machine learning models (deep learning model, or boosting on trees) it tends to be underfitting.
In general, more data* (see comments bellow) means more complex model needed. In other words, if we have huge amount of data, it is better to pick a more complex model. And the simple model such as linear model (for both linear regression and Naive Bayes, see How is Naive Bayes a Linear Classifier? ), will "hit a wall" on performance, i.e., underfitting.
*Here more data means more data points/samples/instances, NOT features/columns.