0
$\begingroup$

I've been learning machine learning, and I've heard that one of the questions to think about when doing machine learning is whether the past (known) data is representative of the future/generalizes well into the future. I've heard that neural networks will work better with this kind of data than data without this property. How exactly do you determine if the data generalizes well?

One thing I've learned in statistics that may be able to do this is when looking for correlation between variables and calculating the p-value of their correlation. If the p-value of this statistic is low and the correlation is considered statistically significant, then it is likely that this statistic did not come about by chance. I was thinking that if this is the case between two variables, then maybe there is a function that could represent their correlation, and thus these two variables might generalize well in the future. Is this line of thought correct?

Also, are there other ways of determining if the data generalizes well?

Improve this question
$\endgroup$
2
  • $\begingroup$ I guess you want to see how you can learn whether a model generalizes well because data is data, and it never changes. So in my understanding, the naive method is to check your model iteratively in as many different datasets as possible and keep optimizing your params. $\endgroup$
    – Memphis Meng
    Commented Apr 24, 2021 at 4:09
  • 1
    $\begingroup$ Correlation tells you nothing about the future. The only way to assess whether data have any relevance to the future is to wait for the future to happen. Of course you can do that analysis retrospectively, as suggested by @Memphis, but even then the best you can do is hope that your analytical results continue to apply at later times. $\endgroup$
    – whuber ♦
    Commented Apr 24, 2021 at 15:42

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.