Not really my area of expertise but I think one answer should be “Nothing!” You could of course try to improve the model or try other techniques (but if you begin tuning the model on the basis of its performance in the test set, you still run the risk of “overfitting”) but changing the size of the test set does not seem to directly address the problem.

If today was still 2011 and we were trying to predict electricity consumption until 2013, this model would give us some seriously misleading predictions. This is precisely the type of things out-of-sample evaluation is supposed to pick up. You can look at it retrospectively today and interpret it as a trend that started last year because of some change to the electricity market but the conclusion remains the same: This model did not allow you to see it coming.

Also, if you read the sentence carefully you will notice that Rob Hyndman also stresses that size of the test set should depend on how far ahead you want to forecast. Intuitively, if you want to predict 24 months, a test set of 7 months is too short, no matter whether you have 100 or 10000 months of past data. For example, a good model of seasonal changes in your data could look very good even if it is unable to predict any year-on-year trend.

Not really my area of expertise but I think one answer should be “Nothing!” You could of course try to improve the model or try other techniques (but if you begin tuning the model on the basis of its performance in the test set, you still run the risk of “overfitting”) but changing the size of the test set does not seem to directly address this problem.

If today was still 2011 and we were trying to predict electricity consumption until 2013, this model would give us some seriously misleading predictions. This is precisely the type of things out-of-sample evaluation is supposed to pick up. You can look at it retrospectively today and interpret it as a trend that started last year because of some change to the electricity market but the conclusion remains the same: This model did not allow you to see it coming.

Also, if you read the sentence carefully you will notice that Rob Hyndman also stresses that size of the test set should depend on how far ahead you want to forecast. Intuitively, if you want to predict 24 months, a test set of 7 months is too short, no matter whether you have 100 or 10000 months of past data. For example, a good model of seasonal changes in your data could look very good even if it is unable to predict any year-on-year trend.