# Linear Regression and Almost Sure Convergence

Consider a linear regression model, wherein: $$y_{i}=x_{i}\beta+\epsilon_{i}$$ where notation is standard and $x$ is a scalar. Let us further impose the following restriction: $$\epsilon_{i}|x_{i}\sim N(0,\sigma^{2})$$

Given mean independence, the OLS estimator $\hat{\beta}_{OLS}$ is both consistent and unbiased. Invoking the concept of almost sure convergence, we have that: $$\Pr\left(\lim_{n\rightarrow\infty}\hat{\beta}-\beta=0\right)=1$$

My question is as follows: $\hat{\beta}$ is distributed exactly as normal, given our normality assumption of the error terms. The normal distribution is continuous, and as a result, the probability of the random variable $\hat{\beta}$ taking on a value of $\beta$ is exactly 0. How can/do I interpret convergence concepts in terms of continuous distributions? Does the above imply that the distribution of $\hat{\beta}$ becomes degenerate over time?

• Your question seems to revolve around a possible misinterpretation of "almost sure convergence" (or possibly in assuming it even applies). The formula you provide is unclear: what exactly do you intend it to say? – whuber Jul 25 '16 at 15:38
• Thanks for your response. I got it from Fumio Hayashi's text "Econometrics".You can find it referenced here: nuffield.ox.ac.uk/teaching/economics/bond/asymptotics1.pdf – ChinG Jul 25 '16 at 15:41
• Please explain his notation, then. There are many different possible meanings one might reasonably attach to your last equation. – whuber Jul 25 '16 at 15:41
• I believe it to be as follows: The probability of the event that the OLS estimate equals exactly the population value as the sample size goes to infinity equals 1. – ChinG Jul 25 '16 at 15:45
• Well, you're correct: the probability that the OLS estimate exactly equals the population value is always zero and so has a limiting value of zero. That can't really mean what your limit expression is intended to say, then. – whuber Jul 25 '16 at 17:38

An equivalent but alternative way to say that: $$Pr(\hat{\beta}−\beta=0)=1$$ as $$n \to \infty$$ Is that: $$Var(\hat{\beta}) \to 0$$ as $$n \to \infty$$
You can see the implications of this on the distribution of $$\hat{\beta}$$. As the sample size increases and approaches the population, the sample essentially becomes "less random". This increases the probability of the estimated parameter being equal to the actual parameter. That is the normal distribution keeps on getting "thinner and taller" as the sample size increases. At the limit of $$n$$, $$\hat{\beta}$$ is no longer random and its distribution does indeed become degenerate. Perhaps where you get mixed up is ignoring that the as we increase the sample size the distribution changes.
• Hi: you can divide by the square root of $n$ and subtract the true ( unknown ) $\beta$ and then use the CLT to show that the distribution of the standardized term converges to a normal distribution with mean zero so it's not degenerate if you scale it correctly. – mlofton Jul 4 '19 at 14:42