3
$\begingroup$

I'm comparing the LR and NB performance on different datasets. And suddenly I am wondering what if we have a big dataset that is infinitely large (at least ensure both models trained to its best??). How would two algorithms work? Usually, LR tends to be overfitting and it requires regularisation. Would in this case it also tend to be overfitting?

Improve this question
$\endgroup$
1
  • 3
    $\begingroup$ I can make an infinitely large training set pretty quickly: take my one $(x,y)$ example and repeat it infinitely. What assumptions are you making about your infinite training data? The distribution will be more important than the quantity. $\endgroup$
    – Arya McCarthy
    Apr 15, 2021 at 6:15

1 Answer 1

Reset to default
2
$\begingroup$

Usually, LR tends to be overfitting and it requires regularisation. Would in this case it also tend to be overfitting?

I would not agree. Linear regression is a linear model, comparing with popular machine learning models (deep learning model, or boosting on trees) it tends to be underfitting.

In general, more data* (see comments bellow) means more complex model needed. In other words, if we have huge amount of data, it is better to pick a more complex model. And the simple model such as linear model (for both linear regression and Naive Bayes, see How is Naive Bayes a Linear Classifier? ), will "hit a wall" on performance, i.e., underfitting.

*Here more data means more data points/samples/instances, NOT features/columns.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.