# Testing significant effect in 2 by 2 factor design on a binary outcome

I have 4 groups each consisting of 1000 recent college graduates, and I am studying employment immediately after graduation. 2 groups took a business class in college, and 2 groups took a math course in college. Group 1 is a control that has taken neither a business class nor a math course in college. I want to conduct a statistical analysis to find if either class was significant in increasing the graduate in obtaining a job immediately after school. Here's the data:

Group 1: No Math, No Business

1000 subjects, 50 employed

Group 2: No Math, Yes Business

1000 subjects, 60 employed

Group 3: Yes Math, No Business

1000 subjects, 75 employed

Group 4: Yes Math, Yes Business

1000 subjects, 100 employed

-

This seems like a classic problem for logistic regression. Rather than specifying these groups, turn math and business coursework into predictors for employment status. This is easy to code up in pretty much whatever statistical software you have around, although you'll want to look into different contrasts (it sounds like you'd want to use treatment contrasts with "no class" set as the reference level).

-

Self-selection problem need to be addressed here. People who choose to take business classes might be more likely to get a job. So, the sample you have is not random, and the inferences you draw might be incorrect. Heckman procedure is used to correct to self-selection bias. I am not sure if it is applicable for discrete dependent variable though.

-
The Heckman procedure has not been extended to any models besides OLS, see dx.doi.org/10.1007/s10940-007-9024-4. While I agree with your comment, given the elementary nature of the question it would probably be better to further elaborate on your statement. Also the OP gives no evidence they have a suitable instrument to estimate the selection bias, which would make the Heckman procedure a moot point (although self selection into treatment is still relevant). –  Andy W May 10 '11 at 3:20
Andy, the Heckman procedure is extended to a variety of models including panel data estimation. Just google search provides many links. –  EEE May 15 '11 at 19:05
I am not aware of any work in which the Heckman estimator has been extended to situations in which the model is not OLS (panel data can still be OLS). Feel free to provide examples, particularly the OP is asking about a binary outcome, so any recommendation should take that into mind. –  Andy W May 15 '11 at 20:48

If @EEE's concern can be addressed and you proceed with an hypothesis test, then rather than logistic regression I'd recommend a chi-square test. For a person fairly new to statistical testing, it'll be dramatically easier to conduct, interpret, and explain to an audience. Plus I think it'll give you just about as much information.

-

One of the more basic approaches you could take is a two-way ANOVA (Page from U Delaware), using Math Class = {Y, N} and Business Class = {Y, N} as your two treatments. You would then perform an analysis of variance on the dependent variable (number of people employed) to determine whether taking a math/business class has an impact on how many people were able to get jobs.

One thing you may want to watch out for, however, are some of the assumptions of ANOVA, which may not be appropriate. You can find the assumptions on this wikipedia page. Normality of the residuals may be questionable if there was no random assignment, in which case, there are non-parametric methods.

-