When it comes to selection of a subset of variables from a big data matrix, such as a gene expression matrix, I have come across some publications that have picked a somewhat arbitrary approach to select for the most informative variables, e.g. filtering data based on N top percent of variance, coefficient of variation, median absolute deviation, or standard deviation.
Which one of these terms mentioned above can provide a less biased selection and is statistically more significant (specially in gene expression data?)
I have seen such filtering criteria (based on variance) as below in a paper (link) but I can not figure out why these filtering steps have been applied. For example, wouldn't it be enough to select a subset of variable only based on n top percent of IQR?
"To enrich highly informative probe sets, we applied three filters: probe sets had to be among the top 2% of all probe sets with regard to their variance, their maximum minus median expression and their 90% quantile minus median expression across all specimens."
Since I am not an statistician I would appreciate someone could explain these in simple words.