can I increase Model Complexity if I get a larger Training Set? This is to double-check a few general ML principles. Could anyone please confirm:
Is the representation in the picture below "generally" correct?

So are the following implications correct? 
1) real life performance is generally much worse than training performance, when I fit the on a small training set. 
2) real life performance is very close to training performance, if the training set is sufficiently large.
3) Simpler models perform generally better than complex models, when the training is small, because they have less chances to overfit.
4) Complex model perform generally better than simpler models, when the training set is sufficiently large. Training set size mitigates overfitting.
5) The larger my training set, the more complex models I can afford to use. So the decision about what is the best model complexity, depends on how much data I have.
Thank you!
 A: I think what you say is generally true for many machine learning problems of interest. For example, trying to categorize an image as x or y (something that is pretty difficult).  In general, a larger training set means that you can increase model complexity.

1) real life performance is generally much worse than training
  performance, when I fit the on a small training set.

Generally, yes.  It depends, though. The problem may be very easy, in which case you can separate all of the cases just based on a very small training set. For example, consider a perfectly separable training set where the test set will be identical. You could potentially learn a perfect model that generalizes perfectly from two examples only.

2) real life performance is very close to training performance, if the
  training set is sufficiently large.

As the training set size increases, the probability that the model approximates the true model increases.  "Real life" examples are drawn from the true model.  So, sure.

3) Simpler models perform generally better than complex models, when
  the training is small, because they have less chances to overfit.

Generally, maybe.  If the decision boundary is nonlinear, a linear model will probably perform worse than a nonlinear model that overfits some.  It is correct that a complex model will overfit more on a smaller training set.

4) Complex model perform generally better than simpler models, when
  the training set is sufficiently large. 

True, if the task is not simple.  If the problem is linearly separable, a simple model will perform as well as a complex model.

Training set size mitigates
  overfitting.

True

5) The larger my training set, the more complex models I can afford to
  use. So the decision about what is the best model complexity, depends
  on how much data I have.

True, but, again, it depends not just on how much but the nature of the data (and, for example, in the case of classification the boundary) as well.  
