How to adjust confounders in Logistic regression? I have a binary classification problem where I apply logistic regression.
I have a set of features that are found significant.
But I understand that Logistic regression doesn't consider feature interactions
While I read online that, it can be accounted by adjusting logistic regression for con-founders
Currently I did this and got the significant features
model = sm.Logit(y_train, X_train)
result=model.fit()
result.summary()

But how do I adjust for confounders? any pointers to tutorial for non-stat person like me would really be helpful
Is there any python package that can help us do this?
Can someone let me know on how can I do this?
 A: The term confounder (or confounding) is used quite loosely these days. Usually, what people mean is to include additional variables into the regression model. In the medical literature, these are often age or gender. I'm not sure how this relates to interactions, which are merely used to allow for non-linear effects in a linear regression model. Consider the logistic regression model
$$
\mathrm{logit}\left( \text{Diabetic} \right) = \beta_0 + \text{Weight} \cdot \beta_1 ,
$$
where the coefficient $\beta_1$ measures the contribution of weight ignoring a person's gender. When adding an interaction with gender, the model becomes
$$
\mathrm{logit}\left( \text{Diabetic} \right) = \beta_0 + \text{Weight} \cdot I(\text{Gender} = \text{Male}) \cdot \beta_1 + \text{Weight} \cdot I(\text{Gender} = \text{Female}) \cdot \beta_2,
$$ i.e., the contribution of weight is allowed to vary across genders.
In contrast, "controlling for the confounder gender", would correspond to the model
$$
\mathrm{logit}\left( \text{Diabetic} \right) = \beta_0 + \text{Weight} \cdot \beta_1 + I(\text{Gender} = \text{Male}) \cdot \beta_2 + I(\text{Gender} = \text{Female}) \cdot \beta_3 ,
$$
i.e., gender just becomes another feature, but it does not interact with weight and there is only a single coefficient for weight for males and females alike.
Defining Confounding
The term confounding is used loosely in the example above and it is not really clear how a confounder is different from any other feature.The causal inference literature defines confounding in mathematical terms. The objective is to estimate the causal effect of a set of variables $X$ on an outcome $Y$ by including a set other variables $S$ in the regression model to account for confounding. Variables in $S$ must satisfy the so-called back-door criterion. Consider the diagram taken from Pearl, 2009:

The back-door criterion has two conditions (Definition 3 in Pearl, 2009):


*

*No element of $S$ is a descendant of $X$

*The elements of $S$ "block" all "back-door" paths from $X$ to $Y$, namely all
paths that end with an arrow pointing to $X$.


Using $S = \{Z_3\}$ would block the path $$X \leftarrow W_1 \leftarrow Z_1 \rightarrow Z_3 \rightarrow Y,$$ but not the path
$$
X \leftarrow W_1 \leftarrow Z_1 
\rightarrow Z_3 \leftarrow Z_2 \rightarrow W_2  \rightarrow Y ,
$$
because conditioning on the collider $Z_3$ opens the path between $Z_1$ and $Z_2$ and thus between $X$ and $Y$. Instead, we would need to include $S = \{Z_1, Z_2, Z_3\}$ in the regression model to control for confounding. Other sets are also possible, e.g. $S=\{W_1, Z_3\}$.
Unfortunately, this requires access to a graphical representation, as above, of how we imagine the world looks like. Often, this is unknown and people tend to include all variables in a regression model they have access to. However, there are certain situations where including additional variables in regression model actually introduces confounding, which otherwise would be absent.
A: A non-statistically solid rule used in epidemiology is called "The 10% rule". It states that when the Odds Ratio (OR) changes by 10% or more upon including a confounder in your model, the confounder must be controlled for by leaving it in the model. If a 10% change in OR is not observed, you can remove the variable from your model, as it does not need to be controlled for. 
EDIT:
What I think you are asking is: 
1.) How to adjust for confounding via analysis 
2.) If interaction is different than confounding. 
An important thing to understand about confounding is that it is generally on a dataset by dataset basis. This works with the 10% rule. Essentially, if the OR of your exposure/outcome relationship does not change by 10% or more after adding the third variable into the model, there is not good enough evidence of confounding to keep it in the model. This is the case even if there is evidence in other literature that the third variable may be a confounder. You could hypothetically put in your model one exposure and 10 extra variables, but if none of the variables change the exposure/outcome relationship (OR), then you should not keep them in the model. Although leaving the variables in the model will control for them, but if it does not change your exposure/outcome relationship, it is unnecessary and will only cloud your conclusions.
This article gives some other methods for addressing confounding, including the 10% rule (Just click on view PDF): Hernan 2002 
Interaction differs from confounding in that it your exposure/outcome relationship is different on different levels of a third variable. Essentially, rather than the third variable influencing the OR like a confounder does, the third variable will have different ORs for different categories. An easy example is using gender (Male/Female). Say you are interested in the effect of cigarette smoking on cancer. You conclude that smoking gives an OR of 3.0 for cancer. However, you wish to examine if age is an interactive term. After entering the interactive term into your model, you find that the OR for cigarette smoking and cancer among males is 2.0 and among females is 4.0. In this case, reporting an OR of 3.0 would be misleading, as there is a clear difference between males in females.
Please let me know if this has clarified any issues you have.
A: I have a question, @sebp:

Accounting for confounding just means adding the confounders as additional columns to X_train


*

*Yes, but doesn't adding these additional columns also increase the dimensionality of your dataset?

*Wouldn't that lead to potential overfitting of your predictive model by adding additional feature columns?

*How do we balance the tension between the need to add these confounder columns with the curse of dimensionality by adding those same confounder columns?

