Doubt about state of Predictor Variable in PRF and its implications [Regression Question Series - Part 1]

Question

Given a population $(X,Y)$ we hypothesize underlying population hasa regression line as follows. The conditional expectation is

$$\begin{aligned} & E(Y|x) = \beta_0 + \beta_1x & \text{PRF (1)} \end{aligned}$$

Including the error $\varepsilon$, the prediction of dependent variable would be

$$\begin{aligned} & Y = E(Y|x) + \varepsilon & \text{Prediction (2)} \end{aligned}$$

which is called simple linear regression model for population.

• Random Variables: $X, Y|x, \varepsilon$

• Parameters $\beta_0, \beta_1, \mu_X, \sigma_X^2, \ \ \mu_{Y|x}, \sigma_{Y|x}^2 = \sigma^2$

Questions:

1. As per my understanding and as also asserted here, the predictor $X$ is fixed and known, not random. So the uncertainity is only on $Y$. So, in this case, shouldn't the equation notations be as follows?

$$\begin{aligned} & E(Y) = \beta_0 + \beta_1x & \text{PRF (3)} \end{aligned}$$

$$\begin{aligned} & Y = E(Y) + \varepsilon & \text{Prediction (4)} \end{aligned}$$

2. If my point 1 above is correct, then the characterization of variables and constants in play would be as follows?

   • Random Variables: $Y, \varepsilon$
   • Unknown Parameters: $\beta_0, \beta_1, \mu_{Y}, \sigma_{Y}^2 = \sigma^2$
   • Known Parameters: $\mu_X, \sigma_X^2 $

3. Gung's take: As gung clarifies further here, assuming X as random complicates situation as below

$$\begin{aligned} & E(Y|x) = \beta_0 + \beta_1(x + \eta) & \text{PRF (5)} \end{aligned}$$

This implies, the standard assumption in our simple linear models (as they do not have that extra uncertainity added to slope), is that $X$ is fixed and known. So PRF is generally, $E(Y)$ not $E(Y|x)$. Is this conclusion right? How is taking $E(Y|x)$ but also $x$ as fixed, justified, as some books seem to follow that?

4. My take: It depends on the experiment. Suppose below is an experiment where $X$ = width of palprebal fissure and $Y$ ocular surface area,OSA (just picked up first one I saw in Devore's).

\begin{array}{|c|c|} \hline i & 1 & 2 & 3 & 4 & 5 \\ \hline x_i & 0.40 & 0.42 & 0.48 & 0.51 & 0.57 \\ \hline y_i & 1.02 & 1.21 & 0.88 & 0.98 & 1.52 \\ \hline \end{array}

• If every time I repeat the experiment, if both $x_i$ and $y_i$ would vary, $X$ is also random and $E(Y|x)$ is more appropriate.
• If every time I repeat the experiment, if $x_i$ would be same and only $y_i$ would vary, $X$ is fixed and known and $E(Y)$ is more appropriate.
  Is this conclusion right?

References (link leads to relevant page):
1. Gujarati's Basic Econometrics,
2. lecture

Next question

Update 1 - Ben's answer summary:
\begin{align} & E(Y|x) = \beta_0 + \beta_1x & \scriptsize{X \text{ is fixed, not RV, in scope of regression} } & \checkmark \\ & E(Y) \neq \beta_0 + \beta_1x & \scriptsize{X \text{ is RV, marginal expectation, out of scope of regression} } \\ & E(Y|x) \neq \beta_0 + \beta_1x & \scriptsize{X \text{ is RV, conditional expectation, out of scope of regression? (awaiting his reply) } }\\ \end{align}

Answer 1

At the outset, it is worth stressing what regression analysis actually is. Regression analysis occurs when you want to find the conditional distribution of a response variable $Y$ conditional on known values of an explanatory vector $\boldsymbol{X}$. Of course, this does not preclude the possibility that you might want to ask about the marginal distribution of $Y$ when $\boldsymbol{X}$ is random, but if you are doing the latter, your problem is one of multivariate analysis, not regression analysis.

So, in view of the fact that this is the scope of regression analysis, it makes sense that the notation in regression analysis would stress the fact that we are conditioning on $\boldsymbol{X}$. This conditioning is explicit in the initial specification of your regression model:

$$Y = \mathbb{E}(Y|x) + \varepsilon.$$

(Note that the error is defined as $\varepsilon \equiv Y-\mathbb{E}(Y|x)$, so the above equation is just a re-arrangement of the definition of the error term.) Since regression is undertaken conditional on the explanatory variables, there are no parameters in the model reflecting uncertainty in these variables. Hence, there are no parameters $\mu_X$ of $\sigma_X$ in the model. The only parameters needed in the model are the parameters that describe the form of the true regression function $\mu(x) \equiv \mathbb{E}(Y|x)$ and the parameters that describe the distribution of the error term.

Many of questions are essentially just asking whether it is okay to be interested in the marginal distribution of the response, when the explanatory variables are random. This is absolutely fine, but that is outside the scope of regression analysis. So if we do this, we don't use a "modified" regression analysis; we just work directly in multivariate analysis where such problems are posed.

---

Question 1: In the specification in equations (3)-(4) you have dropped the conditional notation that shows that the true regression function is conditional on $x$. So either you mean for this conditioning to be implicit (i.e., it is still there, but just not reflected in the notation), or what you are writing is wrong. Indeed, it is immediately obvious that this must be wrong, since the marginal expectation of $Y$ is written as a function of $x$, which is not stated to be a conditioning variable.

Question 2: Your proposal in 1 is not correct, so this doesn't arise. In particular, there is no use for parameters $\mu_X$ and $\sigma_X$.

Question 3: It is entirely possible that you might be interested in the marginal behaviour of $Y$ when the explanatory variables are treated as random. There is nothing at all wrong with asking about this --- it is just that this question is outside the scope of regression analysis.

Question 4: This issue doesn't really depend on the experiment; it depends on what you want to know in your inference. If you are interested in the conditional distribution of the response variable, conditional on the explanatory variables, that is a regression analysis. If you are interested in the marginal distribution of the response variable, incorporating uncertainty in the explanatory variables, that is not longer a regression analysis.