R glm probit regression marginal effects

Question

I am using R to replicate a study and obtain mostly the same results the author reported. At one point, however, I calculate marginal effects that seem to be unrealistically small. I would greatly appreciate if you could have a look at my reasoning and the code below and see if I am mistaken at one point or another.

My sample contains 24535 observations, the dependent variable x028bin is a binary variable taking on the values 0 and 1, and there are furthermore 10 explaining variables. Nine of those independent variables have numeric levels, the independent variable f025grouped is a factor consisting of different religious denominations.

I would like to run a probit regression including dummies for religious denomination and then compute marginal effects. In order to do so, I first eliminate missing values and use cross-tabs between the dependent and independent variables to verify that there are no small or 0 cells. Then I run the probit model which works fine and I also obtain reasonable results:

probit4AKIE <- glm(x028bin ~ x003 + x003squ + x025secv2 + x025terv2 + x007bin + x04chief + x011rec + a009bin + x045mod + c001bin + f025grouped, family=binomial(link="probit"), data=wvshm5red2delna, na.action=na.pass)

summary(probit4AKIE)

However, when calculating marginal effects with all variables at their means from the probit coefficients and a scale factor, the marginal effects I obtain are much too small (e.g. 2.6042e-78). The code looks like this:

ttt <- cbind(wvshm5red2delna$x003,
    wvshm5red2delna$x003squ, wvshm5red2delna$x025secv2, wvshm5red2delna$x025terv2,
wvshm5red2delna$x007bin, wvshm5red2delna$x04chief, wvshm5red2delna$x011rec,
    wvshm5red2delna$a009bin, wvshm5red2delna$x045mod, wvshm5red2delna$c001bin,
wvshm5red2delna$f025grouped, wvshm5red2delna$f025grouped,wvshm5red2delna$f025grouped,
    wvshm5red2delna$f025grouped,wvshm5red2delna$f025grouped,wvshm5red2delna$f025grouped,
wvshm5red2delna$f025grouped,wvshm5red2delna$f025grouped,
wvshm5red2delna$f025grouped) #I put variable "f025grouped" 9 times because this variable consists of 9 levels

ttt <- as.data.frame(ttt)

xbar <- as.matrix(mean(cbind(1,ttt[1:19]))) #1:19 position of variables in dataframe ttt

betaprobit4AKIE <- probit4AKIE$coefficients

zxbar <- t(xbar) %*% betaprobit4AKIE

scalefactor <- dnorm(zxbar)

marginprobit4AKIE <- scalefactor * betaprobit4AKIE[2:20] 

#(2:20 are the positions of variables in the output of the probit model 'probit4AKIE' 
#(variables need to be in the same ordering as in data.frame ttt), the constant in   
#the model occupies the first position)

marginprobit4AKIE #in this step I obtain values that are much too small

Answer 1

Make sure that you're calculating the marginal effects correctly. For a continuous variable, the marginal effect is $\frac{\partial \mathrm{Prob}(y_i=1|\mathbf{\bar{x}},\mathbf{\beta})}{\partial x_j}$. These are typically rather small. However for a dummy variable, thinking about the partial derivative isn't very useful, so instead we define the marginal effect as $\mathrm{Prob}(y_i=1|x_j=1,\mathbf{\bar{x}_{-j}},\mathbf{\beta})-\mathrm{Prob}(y_i=1|x_j=0,\mathbf{\bar{x}_{-j}},\mathbf{\beta}))$

In the context of the Probit regression model, the marginal effect for a continuous variable $x_j$ is $\phi(\mathbf{\bar{x}}'\mathbf{\beta})\beta_j$ On the other hand, the marginal effect for a dummy variable $x_j$ is $\Phi(\mathbf{\bar{x}}_{j1}'\mathbf{\beta})-\Phi(\mathbf{\bar{x}}_{j0}'\mathbf{\beta})$ where $\mathbf{\bar{x}}_{j1}$ is $\mathbf{\bar{x}}$ except the $j$'th element is replaced by 1 and $\mathbf{\bar{x}}_{j0}$ is the same except the $j$'th element is replaced by 0.

If you accidentally assume that a dummy variable is continuous to calculate it's marginal effect, you'll probably get values that are much smaller than they should be or what you're expecting, so this may be what happened.

Answer 2

I think there are at least 3 types of marginal effects for a non-linear model like the probit. Authors are often unclear about which one they are using. The marginal effect at the mean (MEM) is what Matt Simpson described. That's the derivative evaluated at the mean values of the explanatory variables. There is also the average marginal effect: $AME=\frac{1}{N}\sum_{i=1}^{N}\frac{\partial \Pr \left( y_{i}=1|x_{1},...,x_{k}\right) }{\partial x_{j}}=\frac{1}{N} \sum_{i=1}^{N}\phi \left( \alpha +\beta _{1}x_{1i}+...+\beta _{k}x_{ki}\right) \beta _{j}$, where $\phi$ is the standard normal density function. That is the average of derivatives for everyone in your sample. There is also the marginal effect at a representative value (MER), which is kind of like the MEM, but the values of explanatory variables are chosen to be representative, perhaps means or medians for continuous and modes for categorical. For dummy variables, the logic is analogous, but you should use the approach written up by Matt. If you got the dummy version working, you can also treat continuous variables the same way. Here's one's an example of how to do this with the dummies with R. Here's one for continuous variables.