basic questions about expected value [closed]

I'm trying to learn machine learning and I'm filling in the gaps in my knowledge as I go along. I see from this definition that

$$E[X] = \int_{\mathbb{R}} xf(x) dx$$

But what is $$E[\hat{\beta}|X]$$? It is defined here. Is it expected value of $$\hat{\beta}$$ given $$X$$?

• It is a conditional expectation. Heuristically, the value you expect $\hat\beta$ to take when you know $X$. Non heuristically, you can also study measure theory to learn about conditional expectation rigorously...might take a few years though. – Christoph Hanck Oct 5 at 10:16

Yes. it is the expected value of the estimator $$\hat\beta$$ of $$\beta$$ given your data $$X$$.

EDIT:

For example, consider the model $$y = X\beta+\epsilon$$ with $$E[\epsilon|X] = 0$$.

The ordinary least squares estimator of $$\beta$$ is $$\hat{\beta} = (X'X)^{-1}X'y$$, therefore you can calculate its conditional expectation as

$$E[\hat{\beta}|X] = E[(X'X)^{-1}X'y|X] = (X'X)^{-1}X'E[y|X] = \beta$$

This happens because a random variable $$X$$ conditioned on itself is a constant and you can take it out of the expectation.

• Thanks. I have a follow up question. How may one estimate/calculate $E[\hat{\beta}|X]$ if one is given $X$ and corresponding observations $y$? – Nachiket Oct 5 at 10:35
• I can see how one can calculate $\hat{\beta}$ using $\hat{\beta} = (X^TX)^{-1}X^Ty$ but can't see how to calculate $E[\hat{\beta}|X]$ – Nachiket Oct 5 at 10:43
• Do you get now how to calculate it? – Ale Oct 5 at 10:50
• No, I don't get it. I'm guessing that $E[y|X]=X\beta$ and therefore the math works out. But I cannot make sense of $E[\hat{\beta}|X]=\beta$. I would have expected $E[\hat{\beta}|X]$ to be a particular vector, not the original variable ($\beta$) itself. Could you point be to a very basic text on this? My background is in engineering and solid mechanics. – Nachiket Oct 5 at 11:24
• $E[y|X] = E[X\beta+\epsilon|X] = X\beta$ as stated by assumption of $E[\epsilon|X] = 0$. The rest is linear algebra. – Ale Oct 5 at 11:34