1
$\begingroup$

I'm confused about the following four expression used to represent the regression. My intuitions are the following: First: $$Y=Xb+e$$ This is the general linear regression, where random variables $Y$ is explained by a regressors random vector $X$ and $e$ is the difference between $Y$ and its conditional expectation given $X$. Second: $$Y_i=X_ib+e_i$$ This is the representation of linear regression for a random sample. Where the equation represents an equation system with $n$ equations ($n$ is the size of random sample). $X_i$ and $Y_i$ still are random variables, but represent the random experiment associated to select an individual. Thus, $X_i$ and $Y_i$ aren't realizations yet. Third: $$y_i=x_ib+e_i$$ Here $x_i$ and $y_i$ aren't random variables, now they are realizations of the random variables $X_i$ and $Y_i$. Fourth:$$\hat{\ y_i}=x\hat{\ b}$$ This is the estimated model. Where $\hat{\ y_i}$ and $\hat{\ b}$ was calculated based on the estimator obtained from the second equation. My question: are my intuitions correct?

$\endgroup$
2
  • 1
    $\begingroup$ In the last equation, there should be $\hat{y}_i = x_i \hat{b}$, otherwise indices do not correspond, assuming that index $i$ is for observations. $b$ is then a vector of regression coefficients. $\hat{b}$ is calculated by least squares and by multiplying $\hat{b}$ by $x_i$ you get $\hat{y}_i$ . $\endgroup$
    – pikachu
    Aug 3, 2020 at 5:55
  • $\begingroup$ yes, it's a typo. $\endgroup$ Aug 3, 2020 at 19:30

1 Answer 1

0
$\begingroup$

Yes. Your intuition is correct.

Let's start with the first and third examples.
Let be $X$ and $Y$ two random variables.
A linear model that relates $X$ to $Y$ is written as

$$ Y=X\beta+\epsilon $$

This is expressing the linear relationship between the two random variables.

If you have two samples of $X$ and $Y$, $X=\{x_1, x_2, \ldots, x_N\}$, $Y=\{y_1, y_2, \ldots, y_N\}$, the sample linear regression becomes:

$$ y_i = x_i \beta + \epsilon, \quad \forall i \in \{1, 2, \ldots, N\} $$

This is what one uses to estimate $\beta$, for example.

Example 2. If you have a set of $M$ pairs of random variables $\{(X_1, Y_1), (X_2, Y_2), \ldots, (X_M, Y_M)\}$, then you can express the linear relationship for each of them as:

$$ Y_i=\beta_i X_i + \epsilon_i, \quad i \in \{1, 2, \ldots, M\} $$

Bear in mind that in general, $\beta$ can be different from one model to another, that's the reason why I wrote $\beta_i$. In the scheme of individuals, as you described, these are called random effects (for hierarchical models).

Example 4. After estimating $\beta$, denoted $\hat{\beta}$, the prediction from the linear model becomes

$$ \hat{Y} = X \hat{\beta} $$

or

$$ \hat{y}_i = x_i \hat{\beta}, \quad i \in \{1, 2, \ldots, N\} $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.