High accuracy during cross validation, low accuracy on test set I'm currently trying to build a tennis prediction model. Unfortunately, I have some issues that I hope you could help me to handle.
I have 1110 examples of matches from the year 2013, with their outcomes and (which seems to me) relevant features.
I trained a SVM (e1701 package, R) on this training set with a 10 fold cross validation. The model does pretty well: the average accuracy on the 10 fold cv is about 81% of correct predictions.
To further test the model, I tried a "manual CV": I kept a part of the training set out, so I have a test set. I trained the model on the training set, with 10 fold CV, then tested it on the test set. The accuracy is about 81% too. I repeated this process a lot of time, with different sizes of test set and different part of the data set to be the test set. The accuracy is always about 81%. Sounds good.
But here is the problem: I tested the model on some tournament from the year 2014. And the accuracy is pretty low: about 65%.
I don't understand why... 
In your opinion, where does that difference in accuracy comes from ?
What can I do to handle this problem ?
Does it exists some tools to overcome this kind of issues ?
Thank you for your help.
 A: Probably because an other variable plays a part in the outcome in the year 2014.
It is my understanding that predictive models have a limited lifespan.
For your model, both your training set and the data you're looking to predict are from the 2013 season. For instance, each tennis player is one year older, some players might be past their prime shape, and others might have been reaching it in 2014 or on their way to reach it in later years.
I think the difference in your prediction rate comes from those type of factors.
In the phone industry, a model from Sept 2014 with a 80% good prediction might fall to a 60% good prediction in Jan 2015 and lower the following months.
You need to add more variables into your model.
A: First, one thing that comes to mind is that the tournament data is different - the time effect was already mentioned before. Other possibilities include differently encoded features or unusual conditions.
Another possibility is that you did both the cross validation and the evaluation on the test set incorrectly. I am slighty alarmed by 

I have 1110 examples of matches from the year 2013, with their
  outcomes and (which seems to me) relevant features.

When testing the predictive power of your model it is important to not have the test data affect the model at all. One common and potentially very serious error is too pick features first using all the data. This includes an informal analysis using PCA or even just looking closely at the data.
Feature selection and hyperparameter tuning should always be done using training data only.
