Why do OLS and logistic regression coefficients have opposite sign?

Question

I have data $y$ which is the rate of success in $n$ trials. I also have covariates $X$ that I want to regress against $y$ to understand the relationship between them. I tried 2 different approaches. The first is to take the log of $y$ and apply OLS regression. The second is to apply logistic regression to $y$ directly.

I noticed that one theoretically important coefficient, call it $x_0$, is highly significant in both models but in OLS it is estimated as a strong negative effect and in logistic regression it is a small but positive effect.

How is this possible? The variable is binary and I am comparing the standardized coefficients from both models. My assumption is that $x_0$ either increases the rate or not and it should be of the same sign in both models.

UPDATE

When my only regressor is $x_0$ this doesn't happen, it's only when I add the other covariates (of which there is about $200$). I have pasted the output of the full models below, excluding the estimates of the other covariates.

OLS

Call:
lm(formula = fx, data = fit_data, weights = weights)

Weighted Residuals:
         Min           1Q       Median           3Q          Max 
-5.044177447 -0.616615480  0.070124542  0.692928438  4.539252302 

Coefficients
                 Estimate      Std. Error   t value   Pr(>|t|)    
(Intercept)     -1.33734283e+04  4.75932218e+03  -2.80994 0.00495879 ** 
x0              -1.06636833e-01  2.63420263e-02  -4.04816 5.1773e-05 ***

---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 1.01868241 on 25500 degrees of freedom
Multiple R-squared:  0.59443022,    Adjusted R-squared:  0.591090233 
F-statistic: 177.973843 on 210 and 25500 DF,  p-value: < 2.220446e-16

GLM

Call:
glm(formula = fx, family = "binomial", data = fit_data, weights = weights)

Deviance Residuals: 
          Min             1Q         Median             3Q            Max  
-103.51742894    -3.06309844    -0.24575206     2.27364818   165.10350046  

Coefficients:

                 Estimate      Std. Error    z value   Pr(>|z|)    
(Intercept)      2.83201590e+03  1.31990079e+03    2.14563 0.03190268 *  
x0               1.77428670e-02  1.90814767e-03    9.29848 < 2.22e-16 ***

---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 9542439.111  on 25710  degrees of freedom
Residual deviance: 2224838.019  on 25500  degrees of freedom
AIC: 2340850.46

Number of Fisher Scoring iterations: 8
```

Answer 1

I don't think an entire data presentation is needed to give some intuition behind this phenomenon. While we would expect that a logistic regression and OLS model will, on average, produce parameter estimates (slopes or log-odds ratios) that are similar sign, it's entirely possible they will disagree for a given dataset and analysis.

The most likely issue is the influence of observations in the tails. Supposing we have an X that follows a standard normal density, and a modest positive trend in the risk of outcome for greater values of X. The influence of a single observation at (X=5, Y=0) will be far, far greater in a logistic regression model than in a linear regression model. That is because the influence of such an observation in a logistic model can be arbitrarily high. If the fitted value in the logistic model is 0.9, then the Pearson residual will be $(0-0.9)^2/\left( 0.9*(1-0.9) \right)\approx 9$ compared to the linear model where the residual is $0.81$.

The other issue is overadjustment. Due to non-collapsibility of the odds ratio, if (as you say) "several hundreds" of variables are input to the linear model, and suppose several of those variables are unrelated to the outcome, the tendency will be for the OR of effect to attenuate with larger adjustment. Then it becomes likely that the OR will spontaneously flip effect due to random perturbations along the lines of what's pointed out above - again, in multivariate adjustment, it can be difficult to understand the contribution of a single variable, so using the DF-beta diagnostic is a prudent choice.

Answer 2

This is an artifact of the model used for logistic regression, which is model for one of the two probabilities, which add to 1. If your two values are coded, eg, 0 and 1, then linear regression assigns a positive coefficient to any variable, x, that increases in value as y shifts from 0 to 1. So regression assumes that y increases from 0 to 1.

The logistic transformation can model either the probability that y = 1 or that y = 0, eg logit(P(Y=1)) = b0 + b1x or logit(P(Y=0)) = b0 + b1x.

These two models give opposite signs for b1, but they are the same in absolute value and they have the same estimated variance or standard error so they also have the same p-value. Confidence intervals also have their signs switched. The inferences you make about b1 are the same.

The choice of models, for P(y=1) or P(y=1), is arbitrary as far as the logistic method is concerned, when there are two categories.

The person who wrote the software probably wasn't concerned about whether users would want to compare linear regression with logistic regression, only with having a program that worked well.

I seem to recall that the model SAS originally programmed did just what you have noticed. For many years they refused to make any change. After a lot of complaints they included an option that permitted the user to switch the roles of the two categories that resulted in switching the signs of all the variables.