# Why do descriptive statistics contradict with regression coefficents?

I am estimating a binary logistic regression with L1 norm. According to the regression coefficients, the sign of x1's coefficient is positive. However, the descriptive statistics indicates that the mean of x1 variable for class 0 is greater than that of class 1.

It should be noted that descriptive statistics are calculated before the standardization of independent variables.

• Does your logistic regression contain more than one feature? – Matthew Drury Aug 20 '18 at 6:09
• @MatthewDrury Thanks. Yes, it contains a lot of variables, so I used L1 norm (Lasso) to select some important ones. – ebrahimi Aug 20 '18 at 6:12

This not an issue with L1 norms nor with logistic regression specifically; it will happen with ordinary L2-norm multiple linear regression. The direction of an unconditional relationship can change once you condition on another variable (include an additional feature).

Consider 4 groups, where the $p=\text{Pr}(Y=1)$ is an increasing function of $x$ within each group but where the four groups get lower average $p$ as x increases:

Of course the population $p$'s are not observable in practice; you only observe the proportion of Y's that are 1. However, if you have many replicates at each x-value (say 50 or something), you could actually observe something very like this for the sample proportions at each x..

Now if you fit a logistic to just the y vs x values, ignoring the group factor, you will get a negative coefficient. But as soon as you include group, the coefficient for x will be positive.

[In fact it is possible that the sign could even flip each time you included additional feature.]