9
$\begingroup$

I looked at whether participants recycled after being subjected to a given message. There were three groups: a Control group, group A, and group B.

Since my dependent measure was dichotomous (recycling vs. using the waste bin), I performed a logistic regression in SPSS. However, SPSS only let me compare individual groups to the Control group. I found that group A recycled more than controls, but group B did not.

What I am curious about now is whether group A recycled significantly more than group B.

Is it possible to do such a post-hoc test? If so, how?

Improve this question
$\endgroup$
2
  • $\begingroup$ It sounds like you're looking to run a contrast. $\endgroup$
    – assumednormal
    Mar 30, 2012 at 1:23
  • $\begingroup$ That is true; it is more of a contrast situation than a post-hoc situation, especially given we planned the comparison. But I'd be happy learning about either or both. $\endgroup$
    – Lee
    Apr 2, 2012 at 1:42

3 Answers 3

Reset to default
6
$\begingroup$

Unfortunately, I don't know SPSS. That said, if you want to carry out a Wald test where the null is $H_0: \beta_{groupA} - \beta_{groupB} = 0$ you could ask SPSS the variance/covariance matrix of your parameter estimates and construct the Wald test by hand.

Under $H_0$ your test statistics $\chi^2_{obs}$ is distributed as a $\chi^2$ r.v. with 1 degree of freedom

$$ \chi^2_{obs} = \frac{(\hat{\beta}_{groupA} - \hat{\beta}_{groupB})^2}{{\rm var}[\hat{\beta}_{groupA}]+{\rm var}[\hat{\beta}_{groupB}]-2*{\rm cov}[\hat{\beta}_{groupA},\hat{\beta}_{groupB}]} $$

Now you can calculate your p-value. But I am sure that SPSS has a command to perform specific tests on the estimated parameters.

Improve this answer
$\endgroup$
6
  • 2
    $\begingroup$ +1, this is good information, but it is also worth noting that because the OP looked at the data first & then decided to test this, & b/c this contrast is not orthogonal to the initial one, it would be best to use the Bonferroni correction for all pairwise comparisons, ie, divide $\alpha_{crit}$ by 3-choose-2 (namely, 3). $\endgroup$
    – gung - Reinstate Monica
    Mar 30, 2012 at 19:22
  • $\begingroup$ This sounds like an excellent suggestion, assuming I can get all the info from SPSS. Do you know of any books or articles that detail it more fully? Also, to @gung, we actually planned this comparison (so perhaps it's a misnomer to call it a post-hoc test). Does that mean it's acceptable not to do a Bonferroni correction? It seems that perhaps we still should, but I find the 3-choose-2 suggestion odd, given that SPSS already did two of the comparisons (A-Control and B-Control). $\endgroup$
    – Lee
    Apr 2, 2012 at 1:32
  • 1
    $\begingroup$ Lee, opinions vary on multiple comparisons. I would argue that deciding a-priori to test all pairwise comparisons is equivalent to testing them all a-posteriori, ie, you don't get anything from the a-priori-ness. W/ 3 groups, there are 3 pairwise comparisons (3-choose-2), if SPSS has already done 2 of them & you want to do the third, you should use the Bonferroni correction (ie, only p<.017 is 'significant' & that's true for all 3 pairwise comparisons--the 2 SPSS did & the 1 you did); but remember also that controlling for type I errors increases the probability of type II errors. $\endgroup$
    – gung - Reinstate Monica
    Apr 2, 2012 at 2:21
  • $\begingroup$ Very good point @gung, +1. Also, what do you think about the original 2 comparisons SPSS did? Say I only wanted to do those two and had planned them -- should alpha values still be reduced? $\endgroup$
    – Lee
    Apr 3, 2012 at 0:38
  • 1
    $\begingroup$ That's harder to say. They're still not orthogonal, but I might go with the original alpha, since they'd be a-priori. It depends on what the power is, how much more data it would take to maintain power & hold alpha to an acceptable level. $\endgroup$
    – gung - Reinstate Monica
    Apr 3, 2012 at 2:09
5
$\begingroup$

SPSS only let me compare individual groups to the Control group.

Actually SPSS Logistic Regression has about 6 built-in types of contrasts. One of them (Indicator) compares each group to a control group, which you can specify using the group's number. I.e., among groups numbered 1 through 4 and labeled as North, South, East, and West, "indicator(3)" will set East as the control group. Another type (Deviation) shows how each group's logit deviates from the (unweighted) average group's logit. It's useful to go into either the general Help files or the Command Syntax Reference, also found in Help, to find the definitions for each type. Personally, I find that Deviation and Indicator are all I ever seem to need. Maybe that makes me a minor-leaguer :-)

Improve this answer
$\endgroup$
3
  • $\begingroup$ I've now read a bit more about the Deviation contrast. It sounds interesting, but what does it mean to compare to the experimental effect? I understand the experimental effect to be the result that the groups are different from each other in some way. How, then, will using this contrast let me test whether group A is different from group B, as well as testing whether both are different from the Control group? P.S. If you can recommend any good books that discuss this contrast, I'd love to read more. $\endgroup$
    – Lee
    Apr 2, 2012 at 1:15
  • $\begingroup$ I've just edited my answer. The Deviation comparison is not to the "experimental effect" but to the logit for the unweighted average of the other groups. An SPSS Syntax Guide or Applications Guide will help you understand the various types of contrasts, as will most standard texts on logistic regression. E.g., Klein and Kleinbaum, or Hosmer and Lemeshow. Perhaps also Menard. $\endgroup$
    – rolando2
    Apr 3, 2012 at 3:03
  • $\begingroup$ Thanks. I've tried Menard but found him to be too dry/difficult. I'll have a look at the others. $\endgroup$
    – Lee
    Apr 4, 2012 at 21:22
4
$\begingroup$

In R you can do general multiplicity-adjusted contrasts for logistic regression. See for example the rms package's contrast.rms function, which uses the R multcomp package.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Can you explain what you mean by multiplicity-adjusted contrasts? What kind of contrast is needed to address my issue? I have been impressed by R and always want to learn more but most importantly I need to know more about the way to structure the analysis. $\endgroup$
    – Lee
    Apr 2, 2012 at 0:51
  • $\begingroup$ contrast.rms (called by contrast(f) where f is a fit object created by an rms fitting function) with conf.type='simultaneous' will give you the ordinary P-values for any contrast listed but when you request more than one contrast, the individual confidence intervals for the contrasts are computed so that the family-wise or overall coverage probability is, say, 0.95. The individual limits adjusted in this way are wider than ordinary confidence limits. With contrast() you can request a series of contrasts, the there is a multiplicity adjustment by context. $\endgroup$
    – Frank Harrell
    Apr 4, 2012 at 12:02
  • $\begingroup$ Thanks for this response. I'm still at a point where I wouldn't know how to do this in R, but it's good to know that it can be done. Perhaps once I've learned more about it. What's the best book for learning how to do this type of thing in R? $\endgroup$
    – Lee
    Apr 4, 2012 at 21:23
  • $\begingroup$ Self-serving but my book Regression Modeling Strategies (which uses the Design package; the rms package is used almost identically) goes into general background, but specific to contrasts the authors of the multcomp package have a new book out. The help file for contrast.rms and Predict can also help as can my course notes at biostat.mc.vanderbilt.edu.rms $\endgroup$
    – Frank Harrell
    Apr 5, 2012 at 2:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.