Overfitting is when we have a model which has memorized the training data and does not perform well in real-world cases.
Okay, say that I had some training points which look like this:
What if the red curve was the actual 'real-world' relationship. And I found this exact model through a learning algorithm based on those training observations.
Has my model overfit by definition despite it being the 'real-world' relationship? I am assuming yes, but I just want to make sure.