0
$\begingroup$

Some books write linear regression function in the following way:

$$ Y = a + b \times X + u$$

While others write it in the following way:

$$ Y_i = a + b \times X_i + u_i$$

$ Y $, $ X $, $ Y_i $ and $ X_i $ are all scalars.

Why is it necessary to use index? Are these two equivalent? Does $ Y_i $ and $X_i$ in the second case refer to one particular observation or are they still variables?

In the book that uses the second notation author writes: $ E(Y|X_i) $ is a function of $ X_i $, where $ X_i $ is a given value of X.

How can $ E(Y|X_i) $ be a function if its argument is a specific number? Should not it be a variable? I think this is wrong.

Later, it writes: $ E(Y_i|X_i) $

My guess is that the first way is written as the sum of functions (since random variable is nothing but a function) but I do not understand why is it necessary to index these functions.

$\endgroup$
4
  • $\begingroup$ Look carefully, is the first version a vector notation? E.g. Indicated by bold letters? $\endgroup$
    – Björn
    Apr 30, 2018 at 7:38
  • $\begingroup$ No. They are scalars in both cases. $\endgroup$
    – G.T.
    Apr 30, 2018 at 7:43
  • $\begingroup$ If so, the first notation is likely sloppy, if a standard regression model with multiple i.i.d. Observations is intended. $\endgroup$
    – Björn
    Apr 30, 2018 at 7:45
  • 3
    $\begingroup$ The second notation simultaneously spells out the model and, via the indices, indicates the idea that we draw a sample from the underlying population as described by the model. If one feels that the latter point is clear, one may as well drop the indices, in my opinion. See books by Wooldridge, who offers further discussion. $\endgroup$ Apr 30, 2018 at 8:12

2 Answers 2

0
$\begingroup$

I think you're asking two different but related questions here: one on indices, and one on expectations. I'll answer the question on expectations first, then explain the one on indices.

The two equations are the same, though admittedly this is confusing. This is something that's rarely explained clearly on this topic (at least in econometrics textbooks) so your confusion may well be because it's never been properly explained to you.

It's important to remember the difference between four different things: a random variable, a realisation of that variable, an observation, and a non-random variable. I'll illustrate these differences, using the notation of the first equation.

In some introductory textbooks on linear regression, $a$ and $b$ are taken to be constants, $X$ to be a variable (this is what textbooks mean when they say $X$ is fixed), and $u$ to be a random variable (usually normally distributed with mean zero, and independent of $X$). If then $$Y = a + b X + u,$$ then it follows that $Y$ is a random variable, since anything that is a function of a random variable is also a random variable. In this way, we are treating $Y$ as being random only through the error $u$. We then want to find the parameters of the equation:

$$ E(Y) = a + bX.$$

Note that $X$ has not changed: it is still a variable. This means that $E(Y)$ is a variable, and the equation gives the regression line.

In other textbooks, $X$ is taken to be a random variable. In this case, $Y$ is random both through the randomnesss of $X$ and of $u$. In this case, we want to find the parameters of the equation:

$$ E(Y | X=x) = a + bx.$$

In this case, we needed to de-randomise $X$ in order for $E(Y)$ to not be a random variable. Note that the two equations are mathematically identical: we just had to de-randomise $X$ in the second case.

Regarding the indices, its important to remember that they indicate a particular observation - the ith observation in your n observations - not a realisation of a random variable. Basically, "observation" does not necessarily mean the same thing as "data"/"realisation"/"value", although it can: it depends on the author.

I hope this helps.

$\endgroup$
4
  • $\begingroup$ So for each particular $X_i$, for example $X_5$, $Y_i$ has pdf such that $ E(Y_i) $ in our case $E(Y_5) = a+b \times X_5 $. In other words: $E(Y|X=X_5) = E(Y_5)$ $\endgroup$
    – G.T.
    Apr 30, 2018 at 12:07
  • 1
    $\begingroup$ Ah no, be very careful: this depends whether $X_5$ is random or non-random. If we take it as non-random, we have $E(Y_5)=a+bX_5$, and if we take it as random, we have $E(Y_5|X_5=x_5)=a+bx_5$. In particular, $E(Y|X=X_5)$ does not mean anything. When you say the expected value of $Y$ given $X$ equal to something, that something must be a lowercase $x$. Let me know if you need further clarification. $\endgroup$
    – notebook
    Apr 30, 2018 at 15:51
  • $\begingroup$ Could you explain in more detail why is it a problem to write $ E(Y_5) = a+b \times X_5 $ and X_5 is random? $\endgroup$
    – G.T.
    May 1, 2018 at 16:07
  • 1
    $\begingroup$ Sure. The reason is, any expected value has to be non-random (expected values are non-random by definition) , and so it would not make sense for the RHS of that equation to be random. $\endgroup$
    – notebook
    May 1, 2018 at 18:50
1
$\begingroup$

There are many different notations used in mathematics and statistics (plus, econometrics often also introduces own notation). In general, the regression equation is written as

$$ Y = a + bX + U $$

where $Y$ is a random variable, $a,b$ are parameters, $X$ is a random variable or fixed regressor, and $U \sim \mathcal{N}(0, \sigma^2)$ is a normally distributed random noise (i.e. random variable). Alternatively people often use Greek letters for the unobserved parameters/variables, i.e. $\beta_0,\beta_1$ for $a,b$ and $\varepsilon$ for $U$.

When you see $Y$ in such equation, you usually think of a vector of $n$ random variables $Y = (Y_1,Y_2,\dots,Y_n)$, same with $X$ and $U$. Since the variables are all identical in terms of the model, the indexes are often dropped to simplify the notation. It may also be written in terms of observed datapoints $y_1,y_2,\dots,y_n$ and $x_1,x_2,\dots,x_n$.

Yet another possible notation would be writing multiple regression as something like

$$ \boldsymbol{y} = \mathbf{X}\boldsymbol{\beta} + \boldsymbol{\varepsilon} $$

where $\boldsymbol{y,\varepsilon}$ are vectors of length $n$ and $\boldsymbol{\beta}$ is a vector of length $k+1$, while $\mathbf{X}$ is a $n\times (k+1)$ matrix, where the $+1$ is about column of ones concatenated to $\mathbf{X}$ for the intercept. Some authors prefer to write vectors as $\vec{y}$, others use bold font $\boldsymbol{y}$, but you can see also no special decoration to indicate the vectors.

The major differences in the notation is caused by the fact that some authors focus on manipulating with vectors and matrices, while others on dealing with random variables and the underlying probabilistic model.

$\endgroup$
1
  • $\begingroup$ In the books that I mentioned above, in both cases $Y$ and $X$ are scalars. $\endgroup$
    – G.T.
    Apr 30, 2018 at 12:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.