Let's say you design a CNN model and it starts overfitting at e.g 70% accuracy during training. Does that mean that 70% is the best accuracy that I can get out of that model design even if I add regularization? Or in other words, can regularization improve the performance of a model?
There is no way to tell without knowing more about your situation. Yes, you may have hit a plateau.
- Or there may be hidden structure in your data that will only become apparent with (much) more data.
- Or some of your predictors may need to be predicted themselves, and you may be able to do a better job at that.
- Or you may not have included a hugely relevant predictor that can increase your validation accuracy to 90%, simply because you don't have the domain knowledge.
In general, it is very hard to say whether we have reached the end of the flagpole. See How to know that your machine learning problem is hopeless?
Actually your last question, does regularization improve model performance, is a bit off. regularization is a way to reduce overfitting, not to check if you hit a plateau yet.
As overfitting means your model try to fit even to noise in training data and not the real underlying pattern, the model will likely to perform bad on test data. That’s why regularization helps to improve performance, as it is adding a term to penalise models which are sensitive to noise.