Multiple testing and logistic regression I want to perform a number of univariate regressions with different symptoms (e.g. fever, cough, sneezing) as the response variable and one categorical variable (which is always the same each time) as an explanatory variable - age group (0-4, 5-14, 15-64, 65+). In total there are 18 symptoms so hence, I want to do 18 regressions and want to correct for multiple testing using Holm's method.
For each regression I get 4 p-values - one overall age group p-value and 3 p-values comparing individual age groups to a reference age group (15-64). I am unsure about how to calculate confidence intervals and adjusted p-values using Holm's method in this scenario. Does anybody know how to do this?
 A: The R rms package supports general simultaneous confidence intervals.  The analyst can specify a series of comparisons, and simultaneous adjustment will be done over all the current set of comparisons.  For more information see the rms contrast.rms function, along with Predict and other functions.  Here's an example using ols.  Substitute lrm for ols to do binary or proportional odds logistic regression.
require(rms)
set.seed(1)
n <- 800
treat <- factor(sample(c('drug','placebo'), n,TRUE))
sex   <- factor(sample(c('female','male'),  n,TRUE))
age   <- rnorm(n, 50, 10)
y     <- .05*age + (sex=='female')*(treat=='drug')*.05*abs(age-50) + rnorm(n)
f     <- ols(y ~ rcs(age,4)*treat*sex)
d     <- datadist(age, treat, sex); options(datadist='d')

# show separate estimates by treatment and sex

plot(Predict(f, age, treat, sex='female'))
plot(Predict(f, age, treat, sex='male'))
ages  <- seq(35,65,by=5); sexes <- c('female','male')
w     <- contrast(f, list(treat='drug',    age=ages, sex=sexes),
                     list(treat='placebo', age=ages, sex=sexes),
                     conf.type='simultaneous'))
xYplot(Cbind(Contrast, Lower, Upper) ~ age | sex, data=w,
       ylab='Drug - Placebo')
xYplot(Cbind(Contrast, Lower, Upper) ~ age, groups=sex, data=w,
       ylab='Drug - Placebo', method='alt bars')

A: For confidence interval, I think you can just use the logistic regression outputs.
For adjust p-values, refer to this wiki link: http://en.wikipedia.org/wiki/Holm%E2%80%93Bonferroni_method
To my understanding, you are first test an overall age group, then testing a nested hypothesis. I would recommend you test overall group effects first, then go deeper to test individual groups.
