2
$\begingroup$

Consider a linear regression model, wherein: $$ y_{i}=x_{i}\beta+\epsilon_{i} $$ where notation is standard and $x$ is a scalar. Let us further impose the following restriction: $$ \epsilon_{i}|x_{i}\sim N(0,\sigma^{2}) $$

Given mean independence, the OLS estimator $\hat{\beta}_{OLS}$ is both consistent and unbiased. Invoking the concept of almost sure convergence, we have that: $$ \Pr\left(\lim_{n\rightarrow\infty}\hat{\beta}-\beta=0\right)=1 $$

My question is as follows: $\hat{\beta}$ is distributed exactly as normal, given our normality assumption of the error terms. The normal distribution is continuous, and as a result, the probability of the random variable $\hat{\beta}$ taking on a value of $\beta$ is exactly 0. How can/do I interpret convergence concepts in terms of continuous distributions? Does the above imply that the distribution of $\hat{\beta}$ becomes degenerate over time?

|cite|improve this question
$\endgroup$
  • $\begingroup$ Your question seems to revolve around a possible misinterpretation of "almost sure convergence" (or possibly in assuming it even applies). The formula you provide is unclear: what exactly do you intend it to say? $\endgroup$ – whuber♦ Jul 25 '16 at 15:38
  • $\begingroup$ Thanks for your response. I got it from Fumio Hayashi's text "Econometrics".You can find it referenced here: nuffield.ox.ac.uk/teaching/economics/bond/asymptotics1.pdf $\endgroup$ – ChinG Jul 25 '16 at 15:41
  • $\begingroup$ Please explain his notation, then. There are many different possible meanings one might reasonably attach to your last equation. $\endgroup$ – whuber♦ Jul 25 '16 at 15:41
  • $\begingroup$ I believe it to be as follows: The probability of the event that the OLS estimate equals exactly the population value as the sample size goes to infinity equals 1. $\endgroup$ – ChinG Jul 25 '16 at 15:45
  • 2
    $\begingroup$ Well, you're correct: the probability that the OLS estimate exactly equals the population value is always zero and so has a limiting value of zero. That can't really mean what your limit expression is intended to say, then. $\endgroup$ – whuber♦ Jul 25 '16 at 17:38
-1
$\begingroup$

An equivalent but alternative way to say that: $Pr(\hat{\beta}−\beta=0)=1$ as $n \to \infty$ Is that: $Var(\hat{\beta}) \to 0$ as $n \to \infty$

You can see the implications of this on the distribution of $\hat{\beta}$. As the sample size increases and approaches the population, the sample essentially becomes "less random". This increases the probability of the estimated parameter being equal to the actual parameter. That is the normal distribution keeps on getting "thinner and taller" as the sample size increases. At the limit of $n$, $\hat{\beta}$ is no longer random and its distribution does indeed become degenerate. Perhaps where you get mixed up is ignoring that the as we increase the sample size the distribution changes.

Note that the answer is in rather non technical terms. I hope that someone can provide a more technical explanation of it.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Hi: you can divide by the square root of $n$ and subtract the true ( unknown ) $\beta$ and then use the CLT to show that the distribution of the standardized term converges to a normal distribution with mean zero so it's not degenerate if you scale it correctly. $\endgroup$ – mlofton Jul 4 at 14:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.