3
$\begingroup$

I've already read this: Conditional expectation function

For this question, we assume familiar notation in linear regression, with $Y$ being the response and stochastic regressors $X$. I've seen both $E(Y|X)$ and $E(Y|X=x)$ referred to as the "conditional expectation function".

I understand that $E(Y|X)$ is a random variable while $E(Y|X=x)$ is a realization of $E(Y|X=x)$. That being said, in the regression setup, we view realizations of Y and X, which we denote $(y, x)$.

Therefore, we condition on the fact that $X=x$. Then, we use our data to estimate $E(Y|X=x)$.

My question is when $E(Y|X)$ comes into play at all? Where do we even consider it? Or is it irrelevant, since we've already observed a realization of it?

A great answer would also explain why we call both $E(Y|X)$ and $E(Y|X=x)$ the conditional regression functions, as they seem to be certainly related, yet different objects; one is random while the other is deterministic.

Notice that throughout my answer, we've assumed $X$ is stochastic. Therefore, the question is not a question about stochastic vs fixed regressors.

Improve this question
$\endgroup$
10
  • 2
    $\begingroup$ Is this question about notation (the two are meaning the same) or about understanding the concept? $\endgroup$
    – Sextus Empiricus
    Dec 15, 2020 at 8:09
  • 1
    $\begingroup$ I remember recently seeing a question about this different notation. The answer contained something like P(Y|X) being more used for events or categorical variables like P(sick | runny nose) and the notation P(Y|X=x) is used to stress that X is a numerical variable like P(sick | body temperature) $\endgroup$
    – Sextus Empiricus
    Dec 15, 2020 at 8:09
  • $\begingroup$ I was asking both. If you could share the question you saw in addition to answering the question that would be great too! $\endgroup$
    – user303375
    Dec 15, 2020 at 8:14
  • $\begingroup$ In response to the second comment, the distinction is certainly not between categorical and numerical, at least where I've seen it used. Particularly in this link: timlrx.com/2018/02/26/… $\endgroup$
    – user303375
    Dec 15, 2020 at 8:18
  • 1
    $\begingroup$ That link is very nice +1 for Dilip. I do not disagree so much with the explanation there. Indeed $E(Y|X)$ can be a random variable $E(Y|X=x)$ is not. However, I do believe that you can see it more broad. The notation $X=x$ versus $X$ is not just about discriminating between random variables and specific outcomes. $\endgroup$
    – Sextus Empiricus
    Dec 15, 2020 at 17:34

1 Answer 1

Reset to default
1
$\begingroup$

In linear regression, we assume the random variable $E(Y|X)$ has the parametric form $X\beta$. When we "fit" the model, we are estimating the parameter $\beta$, which according to our assumption gives us a general formula for $E(Y|X)$. If we made no assumptions whatsoever about the form of $E(Y|X)$, then it would be true that we would only be able to estimate $E(Y|X=x)$ for our observed realizations $(x,y)$. But that wouldn't be a linear regression model.

The fact that one might refer to both $E(Y|X)$ and $E(Y|X=x)$ by the name "conditional regression function" is just a case of overloading terminology. The first can be thought of as a function of the random variable X, the second can be thought of as a function of the fixed sample $x$. This conflation happens a lot with random variables--one might refer to "the sum of a two rolled dice", which can be interpreted both as a function of random variables (the dice themselves being random), or a function of the fixed outcome of two rolled dice, which is deterministic.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Here's my understanding: $E(Y|X)$ appears in the context when $X$ has yet to be observed; i.e. Y = X\beta + \epsilon. In a sense, this is the "unconditional" relationship between $Y$, $X$, and $\epsilon$, as we have not observed $X$ quite yet, so $X$ in addition to $\epsilon$ are still random. $E(Y|X=x)$ appears when $X$ has already been observed as $x$. Therefore, this gives the "conditional on X=x" relationship $Y = x\beta + \epsilon$, where only $\epsilon$ is random. Is this what you were getting at? $\endgroup$
    – user303375
    Dec 15, 2020 at 18:33
  • 1
    $\begingroup$ No, I would not refer to $E(Y|X)$ as "unconditional". They both reflect a statistic of $Y$ conditioned on $X$. They differ only in whether we are speaking of this relationship generally, or at one specific value. The unconditional expectation $E(Y)$ is likely completely different from $E(Y|X)$. EDIT: To clarify, $Y$ is a random variable with a distribution. It also has a conditional distribution $Y|X$, which is different unless $Y$ and $X$ are independent. We are interested in calculating a statistic of this conditional distribution. $\endgroup$
    – David Foley
    Dec 15, 2020 at 18:42
  • $\begingroup$ Ah makes more sense then. Let me rephrase. If we have not yet observed $X$ yet, then, $Y = X\beta + \epsilon = E(Y|X) + \epsilon$ where $X$ and $\epsilon$ are random. Once we observe the data, $Y = x\beta + \epsilon = E(Y|X=x) + \epsilon$, where only $\epsilon$ is random. We mainly operate in the second case because we're in the specific situation where $X=x$. It is only because we are assuming that $E(Y|X)$ and $E(Y|X=x)$ are linear in $\beta$ that we are able to construct estimates for $\beta$, and therefore estimate both functions. I think this is correct? $\endgroup$
    – user303375
    Dec 15, 2020 at 18:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.