Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 4 down vote favorite
2

I want to discover which genes are expressed in only one of five treatments. This is my pipeline:

  1. ANOVA between the five treatments
  2. Holm multiple testing correction
  3. Tukey for significant genes discovered in step 2

My question is: Should I also correct Tukey p-values for example multiplying the p-value by the number of significant ANOVA p-values (Bonferroni correction on the number of tests performed), or should I only correct at the level of ANOVA?

Thanks a lot in advance,

Rossella

share|improve this question
@user3614 - the "jerk" response is to do away with p-values, and use a Bayesian answer. Bayes theorem in hypothesis testing automatically does the appropriate corrections you require. Multivariate uniform prior on the effects is the way to go, simulate/integrate (whichever is easier) the effects, then calculate the proportion which fall into the rejection region. Use this as your p-value – probabilityislogic Mar 8 '11 at 8:00
@probabilityislogic, best not to confuse others (or yourself) with proselytizing. – cardinal Mar 8 '11 at 17:59
@user3614, how many genes? How many subjects? Bonferroni is usually extremely conservative in this setting. – cardinal Mar 8 '11 at 18:02
@cardinal - touchee. Although from the question as posed, we aren't sure what question @user3614 is trying to answer. p-values give the answer to "how likely is the data, given the effects"? - the Bayesian answer I described gives the answer to "how likely is the true value of the effect sizes, given the data I have?". I would have a guess that the second answer is more appropriate, requiring a Bayesian approach. – probabilityislogic Mar 9 '11 at 3:23

1 Answer

up vote 2 down vote

This sound like a hierarchical hypothesis structure, in which you might want to:

  1. Find genes expressed in any treatment
  2. Find the treatment that caused the expression of a given gene.

You will typically go about by answering (1) and then (2). For answering (1), you will want to aggregate over treatments at each gene. "How to aggregate?" depends on the error measure you wish to control. FDR control seems more appropriate in a preliminary study than FWE control. For FDR control, you can use a Simes aggregate p-value (see reference) or be conservative and use the maximal p-value over treatments. Once you have a single p-value for each gene, look for expression using the Benjamini-Hochberg procedure.

Once you have your subset of expressed genes, you can address (2) and look for the treatment that activated them. Assuming you are comfortable with FDR control, you can use a B-H procedure again, but remember to penalize for the fact you have selected those genes! This can be done by using a stricter error level within each gene; Just divide your desired error level by the number of selected genes.

Here is a reference for details.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.