Logit - comparison of predicted probabilities

I am analyzing, for two different time periods, the probability that an individual will have outcome Y (=1 or 0) given that an event X has occurred (=1 or 0). A number of demographic variables are also included, such as age, gender, etc.

I am interested in looking at very specific cases (and interpreting the results): I look at differences in Y given X holding all other variables at specific values,and do this for time period=0 and then for time period=1.

I would like to know if there is a way to compare the results along these two time periods. The models have the same variables.

I am using Stata 13 and my code looks similar to this:

svy: logit y x##c.age x##female if period==0
margins, level(90) at(age=50 female=1 x=(0 1))


Then I do the same estimation for period == 1:

svy: logit y x##c.age x##female if period==1
margins, level(90) at(age=26 female=0 x=(0 1))

• Commenting on notation: We typically consider y the dependent variable and x the independent variable. It's not a rule, but you will avoid confusing others trying to help if you stick to the convention. – robin.datadrivers Jan 9 '15 at 16:31
• Another comment on your code unrelated to your question: In Stata with svy, you should not use the traditional if statement, but use subpop. Using if may give you incorrect standard errors. See here: stata.com/manuals13/svysubpopulationestimation.pdf – robin.datadrivers Jan 9 '15 at 17:02
• @robin.datadrivers, whoops! not sure how that happened. will edit. I will also look into subpop, thanks! – Nox Jan 9 '15 at 17:26

1 Answer

Stack the data from the two time periods, as you have done, but don't run them separately for the time periods. Use a dummy for time, and interaction terms as appropriate. Try this:

svy: logit y x##c.age x##female x##period


This will tell you if period is significant, and if it moderates y's effect on x. You can then run your margins statement appropriately. You also have to be careful in interpreting interaction terms in logit models, because of their nonlinearity. See these references for a detailed explanation:

Norton, E. C., Wang, H., & Ai, C. (2004). Computing interaction effects and standard errors in logit and probit models. Stata Journal, 4, 154-167.

This is kind of a contentious area and a bit has been written since 2004, though, so you should do more digging. I do believe the current implementation of margins in Stata takes care of this for you, but it would be good to be aware of the issues.

One other comment, for nonlinear models it can be dangerous to compare coefficients across separate samples. Logit models are sensitive to differences in the dispersion of the underlying latent variable, so if the dispersion or variance is different across the datasets, you may not get valid comparisons of coefficients. This isn't typically a concern with linear regression, but it is in a logit model. See this paper if you have access to Sage journals - if not, reading the abstract may be sufficient to understand it's a problem: Karlson, K. B., Holm, A., & Breen, R. (2012). Comparing Regression Coefficients Between Same-sample Nested Models Using Logit and Probit A New Method. Sociological Methodology, 42(1), 286-313.

• Thanks! I have come across this literature, though still not sure how to interpret. I am actually not taking the margins of the interaction terms themselves. Instead I am only looking at the predicted probabilities for very specific scenarios which may get around that problem. There was a particular part of that literature which was confusing for me - is it claiming that you should never include dummy variables in probits (because these are two distinct groups)? For example, should not include female vs male because there may be errors introduced into the probit? – Nox Jan 9 '15 at 17:25
• Provide a citation for that paper, you may be misinterpreting it. – robin.datadrivers Jan 9 '15 at 17:26
• You still want to run a single model, not two separate models, for the reasons I discussed. You would take the margins of the "period" term, and of y. You may not need the interaction term I put in there (it may not be significant in the logit, but in the marginal effects it could be). – robin.datadrivers Jan 9 '15 at 17:28