enter image description here

I have signed a confidentiality agreement with my university so I have hidden some of the content in the picture, I hope that is okay with the community here.

This is the output of R code with glm regression (Binomial-Logit) with both dependent/independent variables as binary(can take either 0 or 1 value).

As can be seen in my results there is this coefficient sign change while adding more regressors, and I am not sure why is that?

My independent variables are all binary and can take only binary values, but still I have ran the cor() test in R for the matrix which showed no collinearity. I have also checked VIF function in R that also shows no collinearity.

I have searched on internet and they talk about confounding variable( I don't know how to find one and I don't have access to further data from this niche industry).

Can you guys help me interpret these results ? or fix them if you think there is a problem?



First, notice the standard errors on the coefficients. The coefficient after adding gender is still within one-third of a standard error of the coefficient before adding gender. Although the sign has changed, these coefficients are actually quite similar to each other. Also, because the coefficient is much smaller than the standard error, the substantive conclusion doesn't change when adding gender: either way, there isn't enough evidence to claim the focal variable has an effect on the odds of the outcome.

I've argued that you shouldn't take the coefficients on the focal variable seriously, but if you did want to take them seriously, there are a few reasons why the coefficient on a focal variable might change when adding a covariate. In logistic regression, if a covariate is associated with the outcome, its inclusion will often change the coefficient on the focal variable, even if it's not associated with the focal variable. This is the problem of noncollapsibility and is one reason to avoid logistic regression models. If a covariate causes both the focal variable and the outcome, then it is a confounder; its exclusion will yield a biased estimate of the effect of the focal variable, so it must be included to meaningfully interpret the focal variable's effect as causal. Adding a confounder into a logistic regression model will change the model coefficients; the direction and magnitude of the change depend on the pattern and strength of the conditional associations among the covariate, focal variable, and outcome. There's nothing particularly interesting about a sign change; it's just a thing that can happen.

Taking a closer look at your data, it seems that the number of observations also decreased when you added the last two variables, indicating you probably have missingness on those variables and you used casewise deletion (i.e., threw out those observations with missing values). If missingness is related to the outcome or the focal variable, discarding missing observations can bias any estimates from the model. There are other, more valid ways of addressing missing data, but they require some assumptions (which are weaker than the assumption that missingness is unrelated to the outcome). Because of the deletion of cases, I would be extremely cautious in interpreting the last two models; they only have half the number of participants as the other models. This is the most likely reason you are seeing changes in the coefficients.

| cite | improve this answer | |
  • $\begingroup$ Per Guo, Jianhua, and Zhi Geng. "Collapsibility of logistic regression coefficients." (1995), it is necessary and sufficient for a control to be conditionally independent from either the outcome OR the "focal" variable, for collapsibility to hold. So, your assertion that "if a covariate is associated with the outcome, its inclusion will often change the coefficient on the focal variable, even if it's not associated with the focal variable" (my emphasis), and so we have non-collapsibility, is really interesting. I would like to explore that, can you provide some literature? Thanks. $\endgroup$ – Alecos Papadopoulos Aug 3 '19 at 13:55
  • $\begingroup$ Mood (2010, p69) and Clarke (2009, p53) have good and accessible discussions on this matter. They take a different angle from collapsibility but my impression is that it's the same problem. $\endgroup$ – Noah Aug 3 '19 at 18:57
  • $\begingroup$ Thank you for such a broad explanation. It really helped! I have another question @Noah, can you please advice me on how to analyze the same variable using logit regression but with three different fixed effects simultaneously. For the results mentioned above, I was using glm regression and when I added three other variable for the fixed effect (industry, year, and country). These variables were considered as dummy variables and I get results which include coefficient for each of the 36 industries, 8 years and 6 countries. $\endgroup$ – Student Aug 4 '19 at 15:17
  • $\begingroup$ I've tried many other R packages e.g. glmmML and bife but it was of no help in getting the desired results. Finally, I came across FENmlm but I am facing issues with that as well. The problem with FENmlm is: When I use fewer regressors with no missing values, everything works fine. When I add all the regressors (those with missing values as well) I get error even though I am using na.rm=T. I also tried removing all the rows which contained missing values first and then ran the femlm function but still I get the following error: $\endgroup$ – Student Aug 4 '19 at 15:28
  • $\begingroup$ "......[Getting cluster coefficients nber 14] max iterations reached (100). Value Sum Deriv (NR) = nan. Difference = nan. Error in if (ll == (-Inf)) return(1e+308) : missing value where TRUE/FALSE needed " I don't understand how does its clustering and overall FENmlm function works because I am getting the above mentioned error even when there are no missing values...:( I checked the source code for the FENmlm but I didn't get any thing so I could understand why this error is coming? $\endgroup$ – Student Aug 4 '19 at 15:28

Not the answer you're looking for? Browse other questions tagged or ask your own question.