Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a data set with my target gene and more than thousand transcriptional factors somehow correlated with this gene. There is data of these factors in more than 70 variable conditions. What I'm looking to do is use CV to find most useful variables which correlate with my target gene. How could I do that in R please?

share|improve this question
I'm used to 'CV' being used for 'coefficient of variation'; given the potential confusion, it's probably best to spell it out as 'cross-validation' the first time you use it. (Yes, I was able to work it out from your tags, but requiring readers to go tag hunting to resolve a confusion they may not even realize they have is not particularly comprehension friendly) – Glen_b Dec 21 '12 at 21:58
Thanks for notice! :) – tutu Dec 22 '12 at 21:12

CV won't help you with that, at least not directly. Make some model of the data and then use some feature selection algorithm, preferably all-relevant one, for instance RF-ACE or Boruta. And then validate it, possibly with CV.

Actual answer:
Let's start from the beginning.

So, you have some attributes (transcription factor levels) and one decision (likely the gene expression), and want to find attributes that are correlated -- ok, so the first approach is to actually calculate correlations and do some (multiple-testing resistant) test to find significant correlations. This is a somewhat trivial approach but people are doing it -- google for feature/gene selection filters and you'll find tons of algorithms with various adjustments, fixes and refinements.

But you can also step further and use a model of TF-gene iterations; if it should appear accurate, you can hope that the fact that it uses certain attributes means they are important in the original process.
Now, how to execute this? Depends on the model. If it is a statistical model (like lassoed robust regression or carefully pruned and refined tree) all you have to do is to dissect it and extract important features (base of the statistical modelling is to minimize parsimony of the model, so it shouldn't contain any noise).

If the model is an ML model (kNN, RF, SVM, GBM, ...; hopefully more sensitive and robust than statistical modelling/hypothesis-testing approach, but also more dangerous) you cannot directly infer anything from its black-box structure, you need some trick to extract this information. This trick is called either a feature selection wrapper or a feature selection embedded method; again, there are tons of them (many actually involve some cross-validation, and this is likely why you thought it is a good algorithm for this job), usually designed specifically for some modelling method.

Now, some of those methods are minimal-optimal, which aim in increasing model accuracy, and all-relevant, which aim at providing possibly complete subset of important features. The latter aim is more compatible with your aims, so you should use an algorithm from this class.

Finally, whatever method you use, you must validate its output to make sure it does not simply overfit (the more dimensional data you have it is more likely that noise will produce strong correlations at random, and FS is just great at inflating this effect). To this end, leave some data untouched for the whole analysis and use it to test the final model or (better) repeat this in a cross-validation fashion, also assessing stability of the feature selection output.
And be prepared for an option that your data is an unpublishable noise (;

share|improve this answer
Oh! Thank you very much for you exhaustive answer! It helped me a lot but still I am a little bit lost - because I am beginner. Are there any tutorials anywhere to use these methods..? – tutu Dec 22 '12 at 21:11

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.