Need help understanding logistic regression output with seemingly contradictory results after swapping reference group

Question

Some background, each of these predictors are 0, 1 one-hot-encoded categories that represent items in a basket (think e-commerce). Each observation can have multiple 1s. For instance, a single observation can have both x1 and x2 and x3, and not purchase x4, or any other items, resulting in a vector such as (1,1,1,0,0,0,0). The aim is to find the odds of Y occurring when purchasing X1,X2,X3 etc.

I have the following logistic regression odds, where X1 is the reference group:

| -         | pvalue  | odds    |
|-----------|---------|---------|
| X2        | 0       | 1.58781 |
| X3        | 0       | 1.37795 |
| X4        | 0       | 1.31701 |
| X5        | 0.00038 | 1.05357 |
| X6        | 0.00583 | 0.95571 |
| X7        | 0       | 0.5504  |
| INTERCEPT | 0       | 0.45808 |

Based on this, my interpretation could include something such as "Compared to X1, X2 has 1.58 times the odds of the outcome"

However, if I switch the reference group to X2 I get the following:

| -         | pvalue  | odds    |
|-----------|---------|---------|
| X3        | 0       | 1.46834 |
| X4        | 0       | 1.34498 |
| X1        | 0       | 1.2982  |
| X5        | 0       | 1.12634 |
| X6        | 0       | 0.57685 |
| X7        | 0.47621 | 1.01191 |
| INTERCEPT | 0       | 0.43695 |

Now I can say something such as, "Compared to X2, X1 has 1.29 times the odds of the outcome". However, I feel like this contradicts the results, because when X1 is the reference group I also get an increased odds...I would expect by swapping the reference groups, I would get odds that are less than 1 for X1.

Why would something like this occur?

Answer 1

Your results differ because you are actually fitting two different models, one where you adjust for all predictors except for $x_1$, and another where you adjust for all predictors except $x_2$. What's happening is that you are trying to treat the vector of predictors $\{x_1, \ldots, x_k\}$ as dummy variables by assuming that one needs to be omitted from the model. If they were dummy variables, then that would be the right thing to do, because any one $x_i$ is a deterministic function of the remaining $x$'s and your model would not be identified if you adjusted for all variables. However, you indicated in your background that the predictors are indicator variables having no deterministic relationship.

Upshot: you need to adjust for all of the predictors in your model, as in $\mathrm{logit}\Pr(Y=1|x_1, \ldots, x_7) = \beta_0 + \sum_{j=1}^7 x_j \beta_j$. There is an intercept plus a coefficient for each of your predictors. My presumption here is that no predictor can be written as a linear combination of the other predictors.

Also, the way you worded your model interpretation is not quite right. The interpretation of the coefficient estimate corresponding to $x_2$ in your first model would be the following: "For fixed values of $x_3$, $x_4$, ..., $x_7$, the odds of the outcome occurring multiplicatively increase by 1.58 when $x_2$ is in the basket versus when it is not"