I see it is often quoted that the omitted variable bias formula is
$$ \text{Bias}\left(\widehat{\beta_1}\right) = \beta_2 \cdot \text{Corr}\left(X_2,X_1\right) $$
where $\widehat{\beta_1}$ is the estimated coefficient in the biased model, $\beta_2$ is the true coefficient of the omitted variable $X_2$ in the full model.
I am wondering how this is derived generally. Thanks.
Here's an example of how one analyzes such a situation. Suppose that the model
$$E[Y\mid X_1,X_2] = \beta_1 X_1 + \beta_2 X_2$$
holds where $X_1,$ $X_2,$ and $Y$ are random $n$-vectors. If you omit the second variable and use the wrong model (omitting the $X_2$ variable)
$$E[Y\mid X] = \gamma_1 X_1,$$
we may ask what error you might expect in using an estimate of $\gamma_1$ to estimate $\beta_1.$ When you use ordinary least squares regression to estimate $\gamma_1,$ the formula is
$$\hat\gamma_1 = \frac{Y\cdot X_1}{X_1\cdot X_1}.$$
Using the correct model formulation you may compute
$$E[\hat\gamma_1\mid X_1, X_2] = E\left[\frac{Y\cdot X_1}{X_1\cdot X_1}\mid X_1,X_2\right] = E\left[\frac{(\beta_1 X_1 + \beta_2 X_2)\cdot X_1}{X_1\cdot X_1}\mid X_1,X_2\right].$$
Basic properties of expectation (linearity) and conditional expectation (taking out what is known) allow you to simplify the right hand side to
$$E[\hat\gamma_1\mid X_1, X_2] = \beta_1 + \beta_2 \frac{X_2\cdot X_1}{X_1\cdot X_1}.$$
By definition, the (conditional) bias in an estimate is the difference between its expectation and estimand,
$$\text{bias} = E[\hat\gamma_1\mid X_1,X_2] - \beta_1 = \beta_2 \frac{X_2\cdot X_1}{X_1\cdot X_1}.$$
That is a general formula, applicable for fixed $X_i$ or for random $X_i$ where $X_1\cdot X_1$ is almost surely nonzero. Already the result is helpful, because it implies that when $X_1$ is orthogonal to $X_2$ (which is just a way of saying the numerator is zero), the bias is zero; and otherwise it shows that the bias is nonzero and it gives you information about its sign and magnitude.
If, additionally, you arrange for the $X_i$ to be standardized (which means their components sum to zero and the squares of their components sum to unity), the fraction on the right could be called the "correlation" of $X_1$ and $X_2,$ understanding this term to be a shorthand for
$$\operatorname{correlation}(X_1,X_2) = \frac{\sum_{i=1}^n X_{1i}X_{2i}}{\sum_{i=1}^n X_{1i}X_{1i}} = \frac{\sum_{i=1}^n X_{1i}X_{2i}}{1} = \sum_{i=1}^n X_{1i}X_{2i},$$
which is the Pearson correlation of standardized vectors. But please note that this is not the correlation in the sense that $X_1$ and $X_2$ might be random vectors: the bias was computed conditionally and still depends on $X_1$ and $X_2.$ If, for instance, each $X_i$ were an iid sequence of random values drawn from a bivariate distribution, that underlying distribution can have a correlation but it's unlikely to equal the value computed in the bias formula.
