Testing significant effect in 2 by 2 factor design on a binary outcome

Question

I have 4 groups each consisting of 1000 recent college graduates, and I am studying employment immediately after graduation. 2 groups took a business class in college, and 2 groups took a math course in college. Group 1 is a control that has taken neither a business class nor a math course in college. I want to conduct a statistical analysis to find if either class was significant in increasing the graduate in obtaining a job immediately after school. Here's the data:

Group 1: No Math, No Business

1000 subjects, 50 employed

Group 2: No Math, Yes Business

1000 subjects, 60 employed

Group 3: Yes Math, No Business

1000 subjects, 75 employed

Group 4: Yes Math, Yes Business

1000 subjects, 100 employed

Answer 1

This seems like a classic problem for logistic regression. Rather than specifying these groups, turn math and business coursework into predictors for employment status. This is easy to code up in pretty much whatever statistical software you have around, although you'll want to look into different contrasts (it sounds like you'd want to use treatment contrasts with "no class" set as the reference level).

Answer 2

Self-selection problem need to be addressed here. People who choose to take business classes might be more likely to get a job. So, the sample you have is not random, and the inferences you draw might be incorrect. Heckman procedure is used to correct to self-selection bias. I am not sure if it is applicable for discrete dependent variable though.

Answer 3

If @EEE's concern can be addressed and you proceed with an hypothesis test, then rather than logistic regression I'd recommend a chi-square test. For a person fairly new to statistical testing, it'll be dramatically easier to conduct, interpret, and explain to an audience. Plus I think it'll give you just about as much information.

Answer 4

One of the more basic approaches you could take is a two-way ANOVA (Page from U Delaware), using Math Class = {Y, N} and Business Class = {Y, N} as your two treatments. You would then perform an analysis of variance on the dependent variable (number of people employed) to determine whether taking a math/business class has an impact on how many people were able to get jobs.

One thing you may want to watch out for, however, are some of the assumptions of ANOVA, which may not be appropriate. You can find the assumptions on this wikipedia page. Normality of the residuals may be questionable if there was no random assignment, in which case, there are non-parametric methods.

Answer 5

you could use the Agresti-Caffo simultaneous confidence interval or (Simultaneous Score Intervals for Difference of Proportions) to compare differences in proportions (Agresti et al. 2008. Simultaneous confidence intervals for comparing binomial parameters, Biometrics 64, 1270-1275).

The corresponding R code is available in http://www.stat.ufl.edu/~aa/cda/software.html