3
$\begingroup$

I'm having trouble with the statistical idea of a fixed regressor, it seems that our $X_i's$ are not treated as random variables, but we are still able to meaningfully condition $Y$ on them in a way that we couldn't if $X_i$ was just the constant $c$. The fixed regressor seems to be occupying a middle ground between a random variable and a constant. I have scoured the forums for all the questions related to this but have been unable to come to a conclusion.

  • I think part of my confusion might stem from different econometrics books loose definition of the simple regression e.g. some use $Y_i = \beta_0 + \beta_1 X_i + u_i$, and make no mention of $\beta_0 + \beta_1 X_i = E(Y|X_i)$.
  • I think the mixed use between capital letters $Y, X$ and lowercase $y, x$ across texts is also not helping. (Given the very specific random variable vs observed value use in statistics generally).
  • I think i'm also getting slightly confused as to when subscript i refers to a fixed value e.g. $X_i$ vs what i assume is a particular but still random value e.g. $Y_i$

Let's take our hypothetical population regression as $Y_i = \beta_0 + \beta_1 X_i + u_i$, where $\beta_0 + \beta_1 X_i = E(Y|X_i)$ and $X_i$ is fixed, $E(u_i) = E(u_i|X_i) = 0$

1) What does it mean to take conditional expectation on a fixed $X_i$?

  • Are we saying $E(Y|X = X_i)$? i.e. $E(Y|X = x)$ where $X_i = x$ an observed value
  • i.e. $\int_{-\infty}^{\infty} y f_{Y|X}(y|X_i) dy$ where $f_{Y|X}(y|X_i) = \frac{f_{Y, X}(y, X_i)}{f_{X}(X_i)}$
  • But then if we are told our regressor $X$ is non-random how can we give it a distribution?

2) How do we take the expectation of $Y_i$ in this form?

$E(Y_i) = E[E(Y|X_i)+u_i]$

  • If $X_i$ were stochastic we would get $E(Y_i) = E(E(Y|X_i)) + E(u_i) = E(Y)$

  • If $X_i$ is non-stochastic then $E(Y|X_i)$ is just a constant? $E(Y_i) = E(E(Y|X_i)) = E(Y|X_i)$

  • However, Gujarati & Porter's Basic Econometrics 5th edition Page 34-35 goes to great lengths to highlight why this isn't the case.

  • But then in what feels like a contradiction on page 40 footnote 8 (Screenshot below) they mention "Note that $E(Y|X_i)$, once the value of $X_i$ is fixed is treated as a constant."

  • They seem to avoid this issue by taking the conditional expectation rather than non-conditional expectation.

$Y_i = E(Y|X_i) + u_i$

$E(Y_i | X_i) = E(E(Y|X_i)|X_i) + E(u_i | X_i) = E(Y|X_i) + 0$

3) Understanding $Y$ vs $Y_i$

  • Is the random variable $Y$ fully defined by the fandom process $Y = \beta_0 + \beta_1X + u$ of which particular values are written as $Y_i = \beta_0 + \beta_1X_i + u_i$?

  • Is this analogous to how we might say a sample of $Z_1...Z_n$ from a random variable $Z$ but where $Z_1...Z_n$ are still random until they become observed values $z_1...z_n$.

  • If so given that $X$ and hence $\beta_0 + \beta_1X_i$ is completely non stochastic how would we define this probability distribution $Y = f(u; X, \beta_0, \beta_1)$ I don't think this is correct because $X$ isn't really a parameter, but it's also not a random variable? So again i don't know how to deal with the fixed regressor.

  • When authors write $y_i = \beta_0 + \beta_1x_i + u_i$ using lower case $y_i, x_i$ generally are they referring to particular observed values or not e.g. Chapter 2 p.g. 27 Woolridge

enter image description here

$\endgroup$

1 Answer 1

1
$\begingroup$

The $X$s are not treated as random variables.

Take the formulation $Y_i=\beta_0+\beta_1 x_i+U_i$ (I am changing $x$ to lowercase to emphasise it is not a random variable, and $U$ to uppercase to emphasise it is). There are only two random variables - $U_i$ (usually assumed to be a Gaussian$(0,\sigma^2)$, and $Y_i$ - a Gaussian$(\beta_0+\beta_1 x_i, \sigma^2)$ by definition.

But then if we are told our regressor $x$ is non-random how can we give it a distribution?

Example: if you have a random variable $A$ defined as $A=B+2$, where $B\sim \text{Gaussian}(0, 1)$, what is the distribution of $A$?

Now, apply the same logic to $Y_i=\beta_0+\beta_1 x_i+U_i$, keeping in mind what is random, and what is not.

^ This also answers your second question.

Regarding the subscript $i$, you can just think of it as "some value". E.g. if you have $n$ data points total, $i$ can refer to any one of those $n$ points. So $Y_1$ is the first, $Y_2$ the second, ..., $Y_i$ the $i$-th. There's not really such thing as $Y$ in this case, it's more just notational laziness.

As a more concrete example, let's say we're regressing weight $Y$ against height $x$. It is true that $X$ has some distribution. But it doesn't matter what that distribution is, because a linear regression model specifies the distribution of $Y$ given the predictors. So the model says "given a person is 170cm tall, what is their weight? Given a person is 153cm tall, what is their weight?" etc.

$\endgroup$
4
  • $\begingroup$ Thank you for the answer i will digest and get back if i have any comments! $\endgroup$
    – CormJack
    Commented Mar 5 at 9:44
  • $\begingroup$ The thing that is confusing me about your answer, is when you say A = B + 2, isn't that analogy just proving that Y has a distribution? Which I understand. What's confusing me is that we condition on X, i.e. E(Y|X), but if x were a constant this would make no sense, but X is also not a random variable so what is going on? i.e to take the conditional expectation (conditional on X) X must have a distribution for this to make sense... Thanks in advance if you're able to provide any more insight $\endgroup$
    – CormJack
    Commented Mar 7 at 19:20
  • $\begingroup$ Linear models are always modelling $E[Y|X]$, or more explicitly, $E[Y|X=x]$. Since we are conditioning on $X$, this means $X$ is known and not a random variable. It doesn't matter how $X$ is distributed, because in the context of linear modelling $X$ is always known to be a value, and thus can be treated as a constant. $\endgroup$
    – Alex J
    Commented Mar 7 at 22:25
  • $\begingroup$ If we were modelling say $Y_i=\beta_0 + \beta_1 X_i + U_i$ where $X$ was not conditioned on, that would be another case. But $X$ is always known in linear modelling. $\endgroup$
    – Alex J
    Commented Mar 7 at 22:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.