Difference between Greek and Latin letters in OLS

Question

Currently I am learning about the OLS method. In the literature I see different notations for the to be explained variable $y_i$. The different notations have to do with the letters used. In one notation Greek letters are used:

$y_i=\alpha+\beta x_i+\varepsilon_i$

In another notation Latin letters are used.

$y_i=a+b x_i+e_i$

In the literature I found the following:

When we analyse empirical data we do not know ‘true’ values of $\alpha$ and $\beta$, but we can compute estimates a and b from the observed data.

So from this statement I understand that if we have a data set, we can only obtain $a$, $b$ which is an estimation for the true parameter. However, it is still not clear to me which notations are used in which context.

It is extremely helpful if somebody could point out the differences between the notations and in which context each notation is applicable.

Answer 1

Using Greek letters for the true parameters and Latin letters for the empirical ones (estimated, computed from the sample) is a rather common convention. E.g., if sampling from a normal distribution, one would often use $\mu,\sigma$ for the true values of mean and standard deviation, and $m,s$ for the ones calculated from sample $\{x_1,...,x_n\}$. Thus, $\mu,\sigma$ are numbers, whereas $m,s$ are themselves random variables.

It is however advisable to always check beforehand what notation is used in a course or a book, as there is no general rule. E.g., using letters with hats for estimated parameters is also rather common (although one might also wish to distinguish between an empirical mean, $m$, and an estimate of the true mean, $\hat{\mu}$.)

Answer 2

A standard notation is to use Greek letters for the true parameter and to put hats on the Greek letters for their estimates, e.g., $\beta$ for the parameter and $\hat\beta$ for its estimate. Most any deviation from this would be somewhat nonstandard and require a definition by the author.

The more important issue is to understand the difference between the parameter being estimated and the estimation itself. The parameters are values that you posit exist, even if you cannot or will not observe them. Consequently, you collect data to make a guess (estimate) about the parameter, and much of statistics could be considered the science of making good guesses.

Answer 3

From my understanding, there is not much difference in terms of the Greek letters or Latin letters to represent the 'true' value of parameters in OLS. If you're referring to the estimates of these parameters, $\hat{\beta}$ is used more frequently than $b$.

In general, Greek letters are more often used as it's convenient to treat $\alpha$ as $\beta_0$ so that the expression can be written as $Y = \beta^T X$ in multivariate regression. In the world of machine learning, the expression is often written as $Y = w^T X$.