Omitted variable bias: which predictors do I need to include, and why? For a last couple of weeks I've been thinking about OVB (Omitted variable bias) in the context of regression and solution for that (how to avoid this problem). I am acquainted with Shalizi's lectures (2.2), but he is just describing this mathematically.
This week somebody said that it's quite easy - the solution for OVB  is to include all those predictors that control the effect of confounding covariates, not all predictors for dependent variable Y.
I am not sure if this is true and yes, I do feel that I lack of deeper knowledge. 
 A: This is not necessarily wrong, but not always feasible and also not a free lunch.
An omitted variable may cause (see, e.g., the comments below for additional thoughts on the matter) bias if it is both (a) related to the outcome $Y$ and (b) correlated with the predictor $X$ whose effect on $Y$ you are primarily interested in. 
Consider an example: You want to learn about the causal effect of additional schooling on later earnings. Another variable that is most certainly satisfies the conditions (a) and (b) is "motivation" - more motivated people will both be more successful in their jobs (whether they are highly schooled or not) and generally choose to receive more education, as they are likely to like learning, and not find it too painful to study for exams.
So, when comparing earnings of highly schooled and less schooled employees without controlling for motivation, you would likely at least partially not be comparing two groups that only differ in terms of their schooling (whose effect you are interested in) but also in terms of their motivation, so the observed difference in earnings should not only be ascribed to differences in schooling.
Now, it would indeed be a solution to control for motivation by including it into the regression. The likely problem is of course: are you going to have data on motivation? Even if you were to conduct a survey yourself (rather than use say administrative data, that will most likely not have entries on motivation), how would you even measure it?
As to why including everything is not a free lunch: if you have a small sample, including all available covariates may quickly lead to overfitting when prediction is your goal. See for example this very nice discussion.
A: Theoretically, including all relevant predictors eliminates the omitted variable bias. However, it might not always be feasible to include all relevant explanatory variables in your regression (due to unawareness of relevant variables or lack of data).
Regarding the lack of knowledge about the omitted variable bias. There are a couple of good lectures out there on the OVB. Looking around, one of the most comprehensive lectures on the omitted variable bias might be this one:
https://economictheoryblog.com/2018/05/04/omitted-variable-bias
It also includes a section that discusses possible strategies against an omitted variable bias. 
A: Carlos' answer is good in that it addresses a major deficiency in regression modeling practice. The term OVB is very imprecise. Except under atypical mathematical structures, adjusting for other variables will change the effect estimated for a primary regressor. This alone does not mean all such variables should be included in a model. 
The "backdoor criterion" specifically addresses confounding bias. An audience of experts will generally not accept/believe results from models which omit confounding variables from adjustment. This is for good reason. Omitted confounders have led to completely incorrect inference in large confirmatory studies, and further led to policies, drug indications, or media coverage which were costly and damaging. The preferred terminology here is confounding bias, rather than merely OVB. This applies to all types of models, including the most prevalent linear regression.
The second most prevalent (perhaps) model is logistic regression. There is another type of "bias" (perhaps) which arises from logistic models unrelated to confounding. You can change the primary effect by adjusting for variables which are uncorrelated with the primary regressor. This is because of the non-collapsibility of the odds ratio. This arises when the primary exposure has a heterogeneous distribution of covariates underlying the baseline risk of the outcome. The slope of the sigmoid which estimates the "averaged out" accumulation of risk per unit difference in a primary regressor is attenuated. This type of bias arises when the target of inference was individual level risk, rather than population averaged.

In general, the advice to modelers is to adjust for prognostic variables, or variables which, despite being unrelated to the primary regressor, are causally predictive of the outcome. Examples might be in a study on lung cancer and smoking, groups of participants by environmental ambient pollution. Assume for the moment no evidence suggests that differences in regionality satisfies backdoor criteria to confound the smoking-cancer relationship. However, the difference in risk for this environmental exposure substantially predicts risk of lung cancer. Adjusting for environmental exposure more finely stratifies these participants so that the apparent differences between smoking and non-smoking, and cancer risk are apparent.
A very nice description of the difference is found here: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3147074/pdf/dyr041.pdf
A: 
the solution for OVB is
to include all those predictors that control the effect of confounding
covariates not all predictors for dependent variable Y.

Yes, this is correct if you are more precise about it. For identification purposes, you should include the variables that control the effect of confounding and avoid those that open confounding paths or mediate the effect you are trying to measure (if you are interested in the total effect) --- that is, you should include those variables that satisfy the backdoor criterion.  You should not indiscriminately include all predictors of $Y$, if by predictor you mean anything that "predicts" $Y$ --- this could bias your estimate. You can find a gentle, example-based, introduction to the topic in this Crash Course in Good and Bad Controls.
In this same vein, it's worth noticing that  Christoph's answer is not strictly correct:

an omitted variable causes bias if it is both (a) related to the
outcome Y and (b) correlated with the predictor X whose effect on Y
you are primarily interested in

This is not true. Correlational criteria is not necessary nor sufficient to define what a confounder is. This is a common misconception on the definition of confounders, illustrated in this other answer.
Of course, which variables to include that guarantees identification addresses only the matter of getting consistent estimates of the causal quantity of interest. You have many other problems to address, such as the efficiency of your estimate (so you might choose/avoid variables that reduce/increase variance), biases due to misspecification of the functional form etc.
