Return to Answer

Maybe: beware. When you say that 70% accuracy (however you measure it) is good enough for you, it feels like you're assuming that errors are randomly or evenly distributed.

But one of the ways of looking at overfitting is that it happens when a model technique allows (and its training process encourages) paying too much attention to quirks in the training set. Subjects in the general population that share these quirks may have highly-unbalanced results.

So perhaps you end up with a model that says all red dogs have cancer -- because of that particular quirk in your training data. Or that married people between the ages of 24 and 26 are nearly guaranteed to file fraudulent insurance claims. Your 70% accuracy leaves a lot of room for pockets of subjects to be 100% wrong because your model is overfit.

(Not being overfit isn't a guarantee that you won't have pockets of wrong predictions. In fact an under-fit model will have swaths of bad predictions, but...)

Maybe: beware. When you say that 70% accuracy (however you measure it) is good enough for you, it feels like you're assuming that errors are randomly or evenly distributed.

But one of the ways of looking at overfitting is that it happens when a model technique allows (and its training process encourages) paying too much attention to quirks in the training set. Subjects in the general population that share these quirks may have highly-unbalanced results.

So perhaps you end up with a model that says all red dogs have cancer -- because of that particular quirk in your training data. Or that married people between the ages of 24 and 26 are nearly guaranteed to file fraudulent insurance claims. Your 70% accuracy leaves a lot of room for pockets of subjects to be 100% wrong because your model is overfit.

(Not being overfit isn't a guarantee that you won't have pockets of wrong predictions. In fact an under-fit model will have swaths of bad predictions, but with overfitting you know you are magnifying the effect of quirks in your training data.)

No. When you say that 70% accuracy (however you measure it) is good enough for you, it feels like you're assuming that errors are randomly or evenly distributed.

But one of the ways of looking at overfitting is that it happens when a model technique allows (and its training process encourages) paying too much attention to quirks in the training set. Subjects in the general population that share these quirks may have highly-unbalanced results.

So perhaps you end up with a model that says all red dogs have cancer -- because of that particular quirk in your training data. Or that married people between the ages of 24 and 26 are nearly guaranteed to file fraudulent insurance claims. Your 70% accuracy leaves a lot of room for pockets of subjects to be 100% wrong because your model is overfit.

(Not being overfit isn't a guarantee that you won't have pockets of wrong predictions. In fact an under-fit model will have swaths of bad predictions, but...)

Maybe: beware. When you say that 70% accuracy (however you measure it) is good enough for you, it feels like you're assuming that errors are randomly or evenly distributed.

But one of the ways of looking at overfitting is that it happens when a model technique allows (and its training process encourages) paying too much attention to quirks in the training set. Subjects in the general population that share these quirks may have highly-unbalanced results.

So perhaps you end up with a model that says all red dogs have cancer -- because of that particular quirk in your training data. Or that married people between the ages of 24 and 26 are nearly guaranteed to file fraudulent insurance claims. Your 70% accuracy leaves a lot of room for pockets of subjects to be 100% wrong because your model is overfit.

(Not being overfit isn't a guarantee that you won't have pockets of wrong predictions. In fact an under-fit model will have swaths of bad predictions, but...)

No. When you say that 70% accuracy (however you measure it) is good enough for you, it feels like you're assuming that errors are randomly or evenly distributed.

But one of the ways of looking at overfitting is that it happens when a model technique (and its training process) pays too much attention to quirks in the training set. Subjects in the general population that share these quirks may have highly-unbalanced results.

So perhaps you end up with a model that says all red dogs have cancer -- because of that particular quirk in your training data. Or that married people between the ages of 24 and 26 are nearly guaranteed to file fraudulent insurance claims. Your 70% accuracy leaves a lot of room for pockets of subjects to be 100% wrong because your model is overfit.

(Not being overfit isn't a guarantee that you won't have pockets of wrong predictions. In fact an under-fit model will have swaths of bad predictions, but...)

No. When you say that 70% accuracy (however you measure it) is good enough for you, it feels like you're assuming that errors are randomly or evenly distributed.

But one of the ways of looking at overfitting is that it happens when a model technique allows (and its training process encourages) paying too much attention to quirks in the training set. Subjects in the general population that share these quirks may have highly-unbalanced results.

So perhaps you end up with a model that says all red dogs have cancer -- because of that particular quirk in your training data. Or that married people between the ages of 24 and 26 are nearly guaranteed to file fraudulent insurance claims. Your 70% accuracy leaves a lot of room for pockets of subjects to be 100% wrong because your model is overfit.

(Not being overfit isn't a guarantee that you won't have pockets of wrong predictions. In fact an under-fit model will have swaths of bad predictions, but...)