Linear Regression Function Notation

Question

Some books write linear regression function in the following way:

$$ Y = a + b \times X + u$$

While others write it in the following way:

$$ Y_i = a + b \times X_i + u_i$$

$ Y $, $ X $, $ Y_i $ and $ X_i $ are all scalars.

Why is it necessary to use index? Are these two equivalent? Does $ Y_i $ and $X_i$ in the second case refer to one particular observation or are they still variables?

In the book that uses the second notation author writes: $ E(Y|X_i) $ is a function of $ X_i $, where $ X_i $ is a given value of X.

How can $ E(Y|X_i) $ be a function if its argument is a specific number? Should not it be a variable? I think this is wrong.

Later, it writes: $ E(Y_i|X_i) $

My guess is that the first way is written as the sum of functions (since random variable is nothing but a function) but I do not understand why is it necessary to index these functions.

Answer 1

I think you're asking two different but related questions here: one on indices, and one on expectations. I'll answer the question on expectations first, then explain the one on indices.

The two equations are the same, though admittedly this is confusing. This is something that's rarely explained clearly on this topic (at least in econometrics textbooks) so your confusion may well be because it's never been properly explained to you.

It's important to remember the difference between four different things: a random variable, a realisation of that variable, an observation, and a non-random variable. I'll illustrate these differences, using the notation of the first equation.

In some introductory textbooks on linear regression, $a$ and $b$ are taken to be constants, $X$ to be a variable (this is what textbooks mean when they say $X$ is fixed), and $u$ to be a random variable (usually normally distributed with mean zero, and independent of $X$). If then $$Y = a + b X + u,$$ then it follows that $Y$ is a random variable, since anything that is a function of a random variable is also a random variable. In this way, we are treating $Y$ as being random only through the error $u$. We then want to find the parameters of the equation:

$$ E(Y) = a + bX.$$

Note that $X$ has not changed: it is still a variable. This means that $E(Y)$ is a variable, and the equation gives the regression line.

In other textbooks, $X$ is taken to be a random variable. In this case, $Y$ is random both through the randomnesss of $X$ and of $u$. In this case, we want to find the parameters of the equation:

$$ E(Y | X=x) = a + bx.$$

In this case, we needed to de-randomise $X$ in order for $E(Y)$ to not be a random variable. Note that the two equations are mathematically identical: we just had to de-randomise $X$ in the second case.

Regarding the indices, its important to remember that they indicate a particular observation - the ith observation in your n observations - not a realisation of a random variable. Basically, "observation" does not necessarily mean the same thing as "data"/"realisation"/"value", although it can: it depends on the author.

I hope this helps.

Answer 2

There are many different notations used in mathematics and statistics (plus, econometrics often also introduces own notation). In general, the regression equation is written as

$$ Y = a + bX + U $$

where $Y$ is a random variable, $a,b$ are parameters, $X$ is a random variable or fixed regressor, and $U \sim \mathcal{N}(0, \sigma^2)$ is a normally distributed random noise (i.e. random variable). Alternatively people often use Greek letters for the unobserved parameters/variables, i.e. $\beta_0,\beta_1$ for $a,b$ and $\varepsilon$ for $U$.

When you see $Y$ in such equation, you usually think of a vector of $n$ random variables $Y = (Y_1,Y_2,\dots,Y_n)$, same with $X$ and $U$. Since the variables are all identical in terms of the model, the indexes are often dropped to simplify the notation. It may also be written in terms of observed datapoints $y_1,y_2,\dots,y_n$ and $x_1,x_2,\dots,x_n$.

Yet another possible notation would be writing multiple regression as something like

$$ \boldsymbol{y} = \mathbf{X}\boldsymbol{\beta} + \boldsymbol{\varepsilon} $$

where $\boldsymbol{y,\varepsilon}$ are vectors of length $n$ and $\boldsymbol{\beta}$ is a vector of length $k+1$, while $\mathbf{X}$ is a $n\times (k+1)$ matrix, where the $+1$ is about column of ones concatenated to $\mathbf{X}$ for the intercept. Some authors prefer to write vectors as $\vec{y}$, others use bold font $\boldsymbol{y}$, but you can see also no special decoration to indicate the vectors.

The major differences in the notation is caused by the fact that some authors focus on manipulating with vectors and matrices, while others on dealing with random variables and the underlying probabilistic model.