Is it possible to perform a regression where you have an unknown / unknowable feature variable?

Question

Is it possible to perform a regression where you have an unknown / unknowable feature variable?

Say I have $y_n = a_0 + a_1 x_1 + a_2 x_2 + a_3 x_3$ but I do not / cannot measure the value of the feature variable $x_3$. Can I still perform a regression to ascertain the coefficients $a_i$?

How about if I have some knowledge of the statistics of how $x_3$ is distributed? If I know that $x_3$ is drawn from a Gaussian distribution $\mathcal{N}(0, \sigma^2)$, with known $\sigma$ does this allow me to perform the regression to ascertain the values of $a_i$?

Answer 1

The complete formula for a linear model is (in quasi matrix form)

$$Y=\beta X+\epsilon$$

So we have multiple coefficents for the variables that we are controlling for, and then we have $\epsilon$, which is everything else which we did not explain with our included variables.

In this error term belong all the variables which we did not consider, either because we do not have information for them or because we simply do not know of them (random deviation).

So there is just no way for you to know what in this term belongs to what unknown term.

Answer 2

How about if I have some knowledge of the statistics of how x3 is distributed?

If you do the regression of $y$ on $x_1$ and $x_2$, then if you're willing to make educated guesses how $x_3$ correlates with each of these, you can calculate what these guesses would entail for how the coefficients you estimate would change if you could observe $x_3$ and ran the full regression.

Suppose for instance that $x_3$ isn't correlated with $x_1$. Then

$\alpha_{2, \text{your regression}} =\alpha_{2, \text{full regression}} + \alpha_3 \cdot \frac{cov(x_3, x_2)}{var(x_2)}$

So if $x_3$ is likely to be only weakly correlated with $y$ or $x_1$ and $x_2$ not much would change. And if it is, you can use these omitted-variable-bias formulas to predict how things would change.

Answer 3

It is always possible... but your estimates will be biased in many cases. The most favorable case occurs:
(a) When $x_{3n}$ is not correlated with the other regressors, in this case, regress $y_n$ on $(\iota,x_{1},x_{2})$ and you have unbiased estimates of $a_0,a_1,a_2$ (Frish-Waugh-Lovell theorem)
(b) If in addition to (a) you know $\sigma$ and $x_3 \sim \mathcal{N}(0, \sigma^2)$, then you can even identify $a_3$: draw $N$ iid values for $x_{3n} \sim \mathcal{N}(0, \sigma^2)$ and regress $y_n$ on $(\iota,x_{1},x_{2},x_{3})$.