Is the difference between two MSEs significant? I developed several Elo rankings and used MSEs to compare them on their predictive capacity of the 2018 World Cup. I've been asked to use a statistical test to find if the difference between two of my model's MSE (0,0588 and 0,0580) is significant. 
Do you have any idea which test should I try?
 A: I don't think it's very useful to do a hypothesis test to compare average prediction errors - that seems like a confusing mix of two different statistical philosophies. A more suitable approach would be to decide based on your subject knowledge how much of an improvement in prediction error would be meaningful, and proceed from there. 
Given that this requires subject matter knowledge to evaluate, it's hard for us to tell you whether 0.0008 is a meaningful difference. 
A: It makes little sense to compare predictive accuracies on a single outcome. Method A could be better or worse than method B for a single output just by chance.
If you have an entire dataset of multiple outcomes you have predicted using methods A and B, then you can more meaningfully test whether A or B is significantly better. The standard test for this is the diebold-mariano test. The tag wiki contains more information and pointers to literature.
A: I do not know about Elo rankings, but it should be similar to following example in statistical meaning. 
Suppose $X_i, i=1,...,n$ follow normal distribution $N(\mu,\sigma^2)$, and the purpose is to estimate $\mu$. One estimate is sample mean, another is sample median. 
Are two estimates the same? Obviously,  the answer is NO.
Based on one pair of sample mean and median to compare two estimates get the conclusion which one is better? The answer is it is impossible.
