I can't recall a single neural network based paper that uses K-fold Validation. Is there a reason for this? Are dropout and other forms of regularization enough to prevent over fitting and make K-fold unnecessary? What am I missing?
Are dropout and other forms of regularization enough to prevent over fitting and make K-fold unnecessary?
Dropout and other forms of regularization don't entirely prevent overfitting. You still need to hold out a validation set which isn't seen at training time.
Is there a reason for this?
Neural network models typically take hours, days, or even weeks to train, so it's not as feasible in terms of obtaining enough computational resources to run k-fold validation. Of course when you only train your model once on only one validation set, there is higher variance in your evaluation results. But that is a tradeoff people are willing to make.