Skip to main content
edited tags
Link
Stephan Kolassa
  • 130.7k
  • 22
  • 264
  • 497
Source Link
Yulia V
  • 165
  • 5

How to evaluate the quality of predictions?

Say I have a big sample of data (simple positive numbers), and I want to check my prediction quality by calibrating my "model" (predictor?) based on the initial part of the sample, generating a prediction and comparing it to the real-life outcome. I have (and understand) the measures of the quality of predictions, but I am missing the intervals design.

Say, the data is $[a_0, a_1, a_2, ..., a_k, ..., a_z]$. I would take $[a_0, a_1, a_2, ..., a_k]$ as the known data, generate $[a_{k+1}, ... a_z]$ and compare the quality. How to choose the size of "given" sample and the size of the sample I am trying to predict?

Does it make sense to make multiple predictions of smaller intervals, e.g. predict $[a_{k+1}, ..., a_l]$ using $[a_0, a_1, a_2, ..., a_k]$, then predict $[a_{l+1} ..., a_m]$ using $[a_0, a_1, a_2, ..., a_k ..., a_l]$, etc.? Again, how to design the sizes of the samples?