# Linear Regression and Almost Sure Convergence

Consider a linear regression model, wherein: $$y_{i}=x_{i}\beta+\epsilon_{i}$$ where notation is standard and $x$ is a scalar. Let us further impose the following restriction: $$\epsilon_{i}|x_{i}\sim N(0,\sigma^{2})$$

Given mean independence, the OLS estimator $\hat{\beta}_{OLS}$ is both consistent and unbiased. Invoking the concept of almost sure convergence, we have that: $$\Pr\left(\lim_{n\rightarrow\infty}\hat{\beta}-\beta=0\right)=1$$

My question is as follows: $\hat{\beta}$ is distributed exactly as normal, given our normality assumption of the error terms. The normal distribution is continuous, and as a result, the probability of the random variable $\hat{\beta}$ taking on a value of $\beta$ is exactly 0. How can/do I interpret convergence concepts in terms of continuous distributions? Does the above imply that the distribution of $\hat{\beta}$ becomes degenerate over time?

• Your question seems to revolve around a possible misinterpretation of "almost sure convergence" (or possibly in assuming it even applies). The formula you provide is unclear: what exactly do you intend it to say?
– whuber
Jul 25, 2016 at 15:38
• Thanks for your response. I got it from Fumio Hayashi's text "Econometrics".You can find it referenced here: nuffield.ox.ac.uk/teaching/economics/bond/asymptotics1.pdf Jul 25, 2016 at 15:41
• Please explain his notation, then. There are many different possible meanings one might reasonably attach to your last equation.
– whuber
Jul 25, 2016 at 15:41
• I believe it to be as follows: The probability of the event that the OLS estimate equals exactly the population value as the sample size goes to infinity equals 1. Jul 25, 2016 at 15:45
• Well, you're correct: the probability that the OLS estimate exactly equals the population value is always zero and so has a limiting value of zero. That can't really mean what your limit expression is intended to say, then.
– whuber
Jul 25, 2016 at 17:38

An equivalent but alternative way to say that: $$Pr(\hat{\beta}−\beta=0)=1$$ as $$n \to \infty$$ Is that: $$Var(\hat{\beta}) \to 0$$ as $$n \to \infty$$
You can see the implications of this on the distribution of $$\hat{\beta}$$. As the sample size increases and approaches the population, the sample essentially becomes "less random". This increases the probability of the estimated parameter being equal to the actual parameter. That is the normal distribution keeps on getting "thinner and taller" as the sample size increases. At the limit of $$n$$, $$\hat{\beta}$$ is no longer random and its distribution does indeed become degenerate. Perhaps where you get mixed up is ignoring that the as we increase the sample size the distribution changes.
• Hi: you can divide by the square root of $n$ and subtract the true ( unknown ) $\beta$ and then use the CLT to show that the distribution of the standardized term converges to a normal distribution with mean zero so it's not degenerate if you scale it correctly. Jul 4, 2019 at 14:42