# Difference $Y=Xb+e$, $Y_i=X_ib+e_i$, $y_i=x_ib+e_i$ and $\hat{\ y_i}=x\hat{\ b_i}$

I'm confused about the following four expression used to represent the regression. My intuitions are the following: First: $$Y=Xb+e$$ This is the general linear regression, where random variables $$Y$$ is explained by a regressors random vector $$X$$ and $$e$$ is the difference between $$Y$$ and its conditional expectation given $$X$$. Second: $$Y_i=X_ib+e_i$$ This is the representation of linear regression for a random sample. Where the equation represents an equation system with $$n$$ equations ($$n$$ is the size of random sample). $$X_i$$ and $$Y_i$$ still are random variables, but represent the random experiment associated to select an individual. Thus, $$X_i$$ and $$Y_i$$ aren't realizations yet. Third: $$y_i=x_ib+e_i$$ Here $$x_i$$ and $$y_i$$ aren't random variables, now they are realizations of the random variables $$X_i$$ and $$Y_i$$. Fourth:$$\hat{\ y_i}=x\hat{\ b}$$ This is the estimated model. Where $$\hat{\ y_i}$$ and $$\hat{\ b}$$ was calculated based on the estimator obtained from the second equation. My question: are my intuitions correct?

• In the last equation, there should be $\hat{y}_i = x_i \hat{b}$, otherwise indices do not correspond, assuming that index $i$ is for observations. $b$ is then a vector of regression coefficients. $\hat{b}$ is calculated by least squares and by multiplying $\hat{b}$ by $x_i$ you get $\hat{y}_i$ . Aug 3, 2020 at 5:55
• yes, it's a typo. Aug 3, 2020 at 19:30

Let be $$X$$ and $$Y$$ two random variables.
A linear model that relates $$X$$ to $$Y$$ is written as

$$Y=X\beta+\epsilon$$

This is expressing the linear relationship between the two random variables.

If you have two samples of $$X$$ and $$Y$$, $$X=\{x_1, x_2, \ldots, x_N\}$$, $$Y=\{y_1, y_2, \ldots, y_N\}$$, the sample linear regression becomes:

$$y_i = x_i \beta + \epsilon, \quad \forall i \in \{1, 2, \ldots, N\}$$

This is what one uses to estimate $$\beta$$, for example.

Example 2. If you have a set of $$M$$ pairs of random variables $$\{(X_1, Y_1), (X_2, Y_2), \ldots, (X_M, Y_M)\}$$, then you can express the linear relationship for each of them as:

$$Y_i=\beta_i X_i + \epsilon_i, \quad i \in \{1, 2, \ldots, M\}$$

Bear in mind that in general, $$\beta$$ can be different from one model to another, that's the reason why I wrote $$\beta_i$$. In the scheme of individuals, as you described, these are called random effects (for hierarchical models).

Example 4. After estimating $$\beta$$, denoted $$\hat{\beta}$$, the prediction from the linear model becomes

$$\hat{Y} = X \hat{\beta}$$

or

$$\hat{y}_i = x_i \hat{\beta}, \quad i \in \{1, 2, \ldots, N\}$$