If I'm calculating FDR or q-value for a collection of linear models, should each term be considered separately? I sometimes have the situation where I have from several dozen to over 100 linear models to perform hypothesis tests on. They have the same predictor variables, but different response variables. 
Let's say I have 100 models and each model has four p-values-- one for the intercept, one each for two main effects, and one for the interaction effect. If I want to calculate false discovery rates, should I calculate one set of FDRs based on the 400 p-values, or should I calculate a separate set of FDRs for each term in the model, based on the 100 p-values for that one term? I've been told by a more experienced colleague that it is the latter, but I don't understand why.
In case it matters, usually one of the main effects and its interactions with the other effects is of primary interest, and the other terms are included because they might influence the response and therefore must be taken into account.
 A: I think this is a good question.  Given that there are 400 hypotheses being tested it seems like you would want to consider 400 tests.  But also you have only 100 separate problems for each there is the overall F test for model significance and then at a second stage tests on the individual model parameters.  I would suggest a third way that seems to make sense to me. For each of the 100 problems do p-value adjustments to each of the tests in that problem and then apply fdr to each of the 400 or more tests that you do but based on their adjusted p-values. I do think there is this hierarchy of testing in this situation that shouldn't be ignored and your first approach ignores it. My approach may be the same as your colleagues but it was not specific enough for me to tell.  The p-value adjustment for the individual tests should be based on perhaps a resampling method such as the permutation and bootstrap adjustments of Westfall and Young or by some standard multiple testing procedure or bound.
