Data Mining-- how to tell whether the pattern extracted is meaningful? I am sure that everyone who's trying to find patterns in historical stock market data or betting history would like to know about this. Given a huge sets of data, and thousands of random variables that may or may not affect it, it makes sense to ask any patterns that you extract out from the data are indeed true patterns, not statistical fluke.
A lot of patterns are only valid when they are tested in the samples. And even those that are patterns that are valid out of samples may cease to become valid when you apply it in the real world. 
I understand that it is not possible to completely 100% make sure a pattern is valid all the time, but besides in and out of samples tests, are their any tests that could establish the validness of a pattern?
 A: You could try:


*

*Bagging http://en.m.wikipedia.org/wiki/Bootstrap_aggregating

*Boosting http://en.m.wikipedia.org/wiki/Boosting

*Cross validation http://en.m.wikipedia.org/wiki/Cross-validation_(statistics)

A: One way to test patterns in stock market data is discussed here. A similar approach would be to randomise the stock market data and identify your patterns of interest, which would obviously be devoid of any meaning due to the deliberate randomising process. These randomly generated patterns and their returns would form your null hypothesis. By statistically comparing the pattern returns in the actual data with the returns from the null hypothesis randomised data patterns you may be able to distinguish patterns which actually have some meaning or predictive value. 
A: If you want to know that a pattern is meaningful, you need to show what it actually means. Statistical tests do not do this. Unless your data can be said to be in some sense "complete", inferences draw from the data will always be provisional.
You can increase your confidence in the validity of a pattern by testing against more and more out of sample data, but that doesn't protect you from it turning out to be an artefact. The broader your range of out of sample data -- eg, in terms of how it is acquired and what sort of systematic confounding factors might exist within it -- the better the validation.
Ideally, though, you need to go beyond identifying patterns and come up with a persuasive theoretical framework that explains the patterns you've found, and then test that by other, independent means. (This is called "science".)
