How to prevent overfitting? [duplicate]

Question

I'm aware of the concept of overfitting in Machine Learning. The main advice for dealing with it, usually is regularization.

Is there other practical advice to avoid overfitting?

Answer 1

The main advice for dealing with it, usually is regularization. Is there other practical advice to avoid overfitting?

I thought what you are actually asking is what is the relation between regularization and overfitting.

The answer is that the strategies designed to reduce overfitting or test error are known collectively as regularization. So I thought the short answer to your question is an emphatic "no".

And here are some regularization strategies listed in the Chapter 7 of the Deep Learning book:

1. Parameter norm penalties

2. Norm penalities as constrained optimization

3. Dataset augmentation

4. Noise robustness

5. Semi-supervised learning

6. Multi-task learning

7. Early stopping

8. Parameter tying and parameter sharing

9. Sparse representation

10. Bagging and other ensemble methods

11. Dropout

12. Adversarial training

13. Tangent distance, tagent prop, and manifold tagent classifier