Why is the p value by average marginal effects different than the p value of the coefficients?

Question

In R, when fitting a logit regression model, why is the p value for variable X different when finding its average marginal effect (AME) (using logitmfx) than when finding variable X's p value by its coefficient (using summary(model))?

Please note that by average marginal effects (AME) I mean the average marginal effects across the entire dataset, not the marginal effect at the mean/average of my model. So, in R, I find the AME by logitmfx(model, atmean = FALSE, data = d)

For example, say the p value of X's average marginal effect is 0.0033 while the p value of the variable X's coefficient is 0.0030.

I understand that this may be due to different hypotheses being tested, but how are they different?

I think I'm confused because I don't understand how the AME is different from the coefficient, in the sense that the AME is the effect across the entire dataset, rather than the marginal effect at a certain point. Wouldn't the coefficient then be similar to the AME across the entire dataset since the coefficient is fitted to the entire dataset, not at a certain point?

Thank you for your help.

Answer 1

The average marginal effect and the slope coefficient in a logistic regression are two very different quantities, so the p-values correspond to very different tests. It is often the case that when one is not zero, the other is not zero, and they are typically in the same direction, so it makes sense that the p-values are similar. For this explanation, I'll assume the focal predictor is binary because it's easier to talk about.

The slope coefficient is the expected difference in the log-odds of the event between the reference and non-reference categories of the focal predictor, holding constant the other predictors. It is a conditional effect, meaning it is only useful for comparing the log-odds of the event happening for two units who were identical on the included predictors but differed only on the focal predictor (i.e., the predictor whose coefficient is under consideration). That is, for two units who were identical on all other measured predictors but differed on the focal predictor, the coefficient on the focal predictor is the difference in the log-odds of the event between these two units. In addition to being a conditional estimate, the coefficient is also on the log-odds scale. Although the change in log-odds doesn't depend on the values of the other predictors (unless an interaction term is in the model), the change in probability corresponding to a change in the focal predictor does depend on the values of the other predictors.

The average marginal effect (AME) is the average of the marginal effects across all units in the sample. A marginal effect for one unit is the effect of changing the focal predictor without changing that unit's other predictor values. The AME is, therefore, the average effect of changing each unit's focal predictor values; it's an effect that refers to the average change across the entire sample for a small, sample-wide, hypothetical intervention on the focal predictor. In addition, the AME is typically reported on the probability scale, not the log-odds or odds scale. The AME corresponds to the average change in probability of the event that would occur if the focal predictor was changed for every unit in the sample. As mentioned above, the conditional change in probability depends on the values of the predictors and the focal predictor, but the AME (the marginal change in probability) is just one value averaging across all units.

To get the AME you do something like the following:

1. Consider one unit. Use the estimated logistic regression model coefficient to get their predicted probability $\hat{p}_{0i}$ under the reference level of the focal predictor, and then get their predicted probability $\hat{p}_{1i}$ under the non-reference level of the focal predictor, each time using their observed values on the other predictors. If your estimated logistic regression model is $$\text{logit}(p_i)=\hat{b}_0 + \hat{b}_f X_{fi} + \hat{b}_1 X_{1i}$$ where $X_{fi}$ is the value of the focal predictor for unit $i$, then the predicted probability for unit $i$ under the reference level of $X_{fi}$ is $$\hat{p}_{0i} = \frac{\exp(\hat{b}_0 + \hat{b}_1 x_{1i})}{1 + \exp(\hat{b}_0 + \hat{b}_1 x_{1i})}$$ and the predicted probability for unit $i$ under the non-reference level of $X_{fi}$ is $$\hat{p}_{1i} = \frac{\exp(\hat{b}_0 + \hat{b}_f + \hat{b}_1 x_{1i})}{1 + \exp(\hat{b}_0 + \hat{b}_f + \hat{b}_1 x_{1i})}$$ The coefficient of the focal predictor is what tells you how much the probability changes that for that unit, but the coefficient tells you that change on the scale of the log-odds rather than the probability scale. That is, $\hat{p}_{1i} - \hat{p}_{0i} \ne \hat{b}_f$. Nonetheless, we can get the two predicted probabilities from the model.
2. Do the above for all units. You will find that the difference between the predicted probabilities, $\hat{p}_{1i} - \hat{p}_{0i}$ varies across all units, even though the coefficient on the focal predictor is the same for everyone. Again, this is because though the change in log-odds is constant, the difference in probability depends on the values of the other predictors.
3. Take the average of the probability differences across the sample, $\frac{1}{n}\sum_i \hat{p}_{1i} - \hat{p}_{0i}$. This average is the AME. Clearly, the AME is several steps removed from the slope coefficient. It is valuable when thinking about the average effect of changing the focal predictor for everyone rather than just for one individual, and it's also on the probability scale rather than the log-odds scale, which makes it more interpretable.