What is the practical way to identify the factors that create variation in a data of a dataset? What category does this question fall into? Are there a set of algorithms that can be used for this purpose? Statistical modeling solutions? I googled a lot but no go! The question is so vague that hard to find an answer for! Thanks in advance ...
migrated from stackoverflow.com Sep 26 '11 at 6:12
This question came from our site for professional and enthusiast programmers.
There are many algorithms for trying to identify these properties.
One powerful algorithm that you could look into is principal component analysis (PCA), which attempts to identify an orthogonal set of vectors that account for the maximum amount of variance in a set of data. This has applications in machine learning, data compression, and dimensionality reduction and can be implemented easily with most matrix packages using the SVD.
Another, more powerful technique that's often used is independent factor analysis, which I confess I know very little about. However, the lecture notes I've linked do an excellent job explaining the technique and its applications.
Hope this helps!
The idea of explaining variation is often used in the context of linear regressions - and therefore of ANOVA, which is mathematically very similar. See http://en.wikipedia.org/wiki/Fraction_of_variance_unexplained and the articles linked off it.