I am doing a personal project to see how well does FIFA potential player stats predict the actual overall stat after 3 years.

  1. Meaning, if a player has a potential of 85 in 2015, how accurate should I expect it to be to predict the overall player score in 2018. Should I use R2 for this? Maybe MAPE?
  2. I also want to check if the histogram of errors (potential_2015 - overall_2018) has a normal distribution. Do I need to use Chi-squared for this?
  3. If the prediction is overestimating the player, I would like to know by how much. Should I use Standard Deviation on the errors?

If you have a list of key statistical tests, I would appreciate if you could list them so I can research and learn.

Thank you,

  1. Here is a list of point forecast accuracy measures. The entire textbook is very much recommended. Also relevant:
  2. There is a number of normality tests, the most common is the Shapiro-Wilks test. In 15 years of forecasting, I have never seen anyone test normality of forecasting errors, but you may have reasons for doing so. Note that normality tests address a question we already know the answer to: since your stats are (presumably) nonnegative, your errors are bounded, but the normal distribution is unbounded, so the errors can't be normally distributed. A test may still be useful in assessing whether they are "too" non-normal. As above, this is assuming you have a reason to be interested in this.
  3. You can simply calculate the error per player, or if you have multiple forecasts and actuals for a player, take the mean error. The standard deviation (of what?) will not be very useful.
  • $\begingroup$ The book at this link goes heavily into the theory of forecasting and the different methods. I haven't read the whole thing but I remember the parts of it I did read being pretty good. Even though it's econometrically focused, a lot of the material is general. Of course, take a look before you purchase because there's a lot material on the net and Stephen's answer is useful also. amazon.com/Economic-Forecasting-Graham-Elliott/dp/0691140138/… $\endgroup$
    – mlofton
    Nov 19 '21 at 8:10
  • $\begingroup$ Thank you @Stephan, 1. After checking your resources and going into the rabbit hole I decided to use MAE or RMSE instead of MAPE, as I don't want to put emphasis on negative errors. 2. My goal to test the normality of errors is because it should not follow a bell curve, as my assumption is that the forecast has a bias to overestimate players potentials. Graphically, the histogram should be skew right (error: actual - potential). I'll use Shapiro-Wilks, thank you. 3. I found I can use Mean Bias Error. $\endgroup$
    – Xavier
    Nov 22 '21 at 5:12
  • $\begingroup$ Thanks for the feedback, just one comment: whether the errors are normally distributed or have a bias is not a contradiction, they could be both, e.g., normally distributed with a nonzero mean. It looks like you are mainly interested in whether the forecasts have a nonzero bias. A t-test would be appropriate, without worrying about the normality. $\endgroup$ Nov 22 '21 at 6:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.