Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 0 down vote favorite

I have one categorical independent variable with two levels (Group 1 and Group 2). These groups were measured at two times on 10 dependent variables, therefore, I have 2 sets of 10 variables for Time 1 and Time 2. For illustration sake, lets consider Hit_1 (tested at time 1) and Hit_2 (tested at time 2) These 10 variables are dependent, i.e. performance on one variable depends on performance on another variable (they are actually signal detection variables from a memory task ie hits, correct rejections etc). The 10 variables are all interval. Some of the variables were not normally distributed so I did a square-root transformation.

How can I do the following:

  1. test whether there is an effect for Group 1 (i.e. differences between Hit_1 and Hit_2)?
  2. for Hit_1 test whether there is a significant difference between Group 1 and 2?

For each of the above, how can I perform a multiple comparison analysis with an alpha correction?

share|improve this question
1  
How many statistics courses have you had? This kind of analysis would require you to have had several. Why should anyone help on an urgent basis? You question is actually quite complicated, you did not provide the experimental design, and you dit not say which measurements are made on different subjects or experimental units and which are made on the same unit. – Frank Harrell Sep 3 '11 at 19:16
Hi Frank. Thanks for the post. Sorry if I didn't provide the relevant information before. The purpose of this study was to compare memory between Group 1 and Group 2. I had two memory tasks, each measuring 10 variables (signal detection variables therefore Hits, Correct Rejections, False alarms etc). Both memory tasks measured exactly the same variables, so therefore I have two sets of variables, all interval with units of measurements as frequencies. All subjects were measured on these 10 variables on these two different tasks. – Tanuja Sep 3 '11 at 19:38
My primary research question is to know whether there were differences between the two groups on each of these variables. I have been doing repeated measures ANOVA because each subject was measured twice on the same variable. – Tanuja Sep 3 '11 at 19:38
However, SPSS averages the observation for the two measurments on each variable when testing for between-group differences, and I would ideally like to know if there were group-differences for a particular variable for time 1 and time 2 sperately. I hope this makes it clearer. Please let me know if I missed something. – Tanuja Sep 3 '11 at 19:41

1 Answer

up vote 1 down vote accepted

Assuming the two groups represents two non-overlapping groups of subjects, and all subjects without group are measured at two times, I suggest you turn the problem around to get a perfect multiplicity correction by using binary logistic regression to predict group from the ten measurements. The global null hypothesis tests whether any of the 10 is associate with group, which is another way of saying that there is a group difference for at least one of the 10. Do this whole analysis separately at the two times. Use the likelihood ratio $\chi^2$ test with 10 degrees of freedom. A reference is below.

Peter C. O’Brien. Comparing two samples: Extensions of the t, rank-sum, and log-rank test. J Am Stat Assoc, 83:52–61, 1988. Key: obr88com
Annotation: see Hauck WW, Hyslop T, Anderson S (2000) Stat in Med 19:887-899

share|improve this answer
Thanks, seems like an elegant solution to my problem! I had a look at the paper (a bit hard to follow!), but here is what I have done. I entered Group as the dependent variable in a binary logistic regression with the variables for the first task as covariates using the forward LR method. Is this appropriate and any tips on interpreting the results? Thanks! – Tanuja Sep 4 '11 at 0:41
I completed the analysis and it was actually pretty straightforward. Thanks! If I find that the omnibus test is significant at 10df, how do I find otu which oen of the 10 variables is implicated? – Tanuja Sep 4 '11 at 6:47
I would not emphasize the forward LR method. The global test with 10 d.f. is the one that is adjusted for context (multiplicity). The data are most capabable of telling you that one or more variables are implicated. If you really must look further, look at the 10 partial tests from the model (don't remove any variables) too see for which ones there is evidence of an independent effect on group (or that group effects them indep. of its effect on the others). The one d.f. partial tests can be obtained using ordinary Wald $Z$ tests or by removing 1 var. at a time (later restoring) to get LR. – Frank Harrell Sep 4 '11 at 13:41
Great, that's pretty clear. Thanks for helping me out! – Tanuja Sep 5 '11 at 19:16

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.