1
$\begingroup$

Suppose b represents the OLS estimator, and B the true coefficient in the regression model without intercept

y = Bx + u. 

Under certain assumptions b is unbiased so that

E[b | X] = B. 

Suppose that I want to have an estimator of

E[b | X].

The estimator of B is b. Hence it should be the estimator of this conditional mean. Then

Est. E[b | X] = b

where Est. stands for estimator. What does this tell me? The OLS estimate I obtain with the sample data at hand gives me the estimate of the mean of the sampling distribution of b. So we assume that the sample data at hand is so typical that it produces a b that is an estimate of the mean of the sampling distribution of the estimator. Is this correct?

$\endgroup$
  • $\begingroup$ Firstly, you are asking for the sampling distribution of the mean of the OLS estimator, correct? Which estimator (intercept $\hat{\beta_0}$ or slope $\hat{\beta_1}$? I assume you are inquiring about the latter. In your post, you are merely demonstrating that the OLS estimator(s) are unbiased. Your third line of R code is not correct. $\endgroup$ – compbiostats Feb 8 '18 at 18:46
  • $\begingroup$ The question is about the sampling distribution and about the mean of that. b is for a variable not for a constant which I should have mentioned. My question is not about whether b is unbiased. It is simply about how to estimate the conditional mean of b. I do not use any R code in my question. I just state expressions. Est. stands for estimate. $\endgroup$ – Snoopy Feb 8 '18 at 18:56
  • 1
    $\begingroup$ You might be overthinking this. Since $E[b\mid X]=B$, estimating either quantity amounts to exactly the same thing, no matter how you perform the estimate. And estimating $E[b\mid X]$ as $b$ is no different than, say, estimating the mean of a population ($B$) with the arithmetic mean $b$ of a sample. In light of this, could you explain what you're looking for by "What does this tell me" and "What does this mean"? $\endgroup$ – whuber Feb 8 '18 at 19:04
  • $\begingroup$ One assumption of an OLS regression is that your estimated expectation 'b' is at the center of a normal distribution defined by X belongs to a normal distribution. Inherent randomness in the underlying process (i.e. the random normal error term in the equation governing the underlying process), means that the samples may not always fall on the expectation, but that if the error in the process is normal, then so too must be the estimated expectation values. $\endgroup$ – rmrouse88 Feb 8 '18 at 19:08
  • $\begingroup$ My question is simply how to estimate the mean of the sampling distribution of b? I deduce in my question that the answer is b. What is the estimator of Var[b | x]? It is s^2 (x'X)^{-1}. So what is the estimator of E[b | X]? It is b. Right? I am just questioning what this tells. So I have sample data at hand. I calculate b, and this b gives me an estimate of the mean of the sampling distribution of b. Is this correct? $\endgroup$ – Snoopy Feb 8 '18 at 21:21
1
$\begingroup$

Your surprise might stem from a reasoning in a different direction, why is $\hat \beta$ not only estimate for $\beta$ but also for the mean of it's own sampling distribution $\mu_{\hat\beta}=E(\hat \beta)$.

But you always get (by default) for any random variable $X$ with finite mean, that a particular value $x$ sampled from the distribution of $X$ is an unbiased estimate for $E(X)$.

  1. a single $b$, drawn from the distribution for $B$ is an unbiased estimator for $E(B)$
  2. if $E(B)=\beta$, then $b$ is (also) an unbiased estimate for $\beta$

The property 1 is: not some additional rule to 2, not a consequence of 2, and neither a consequence of OLS estimation.

You always have 1, and sometimes 2.

Your line of thought goes like we have 2, but also 1, why is that? The answer is a bit trivial: since you always have 1.


Where:

  • $B$ is a variable and more specifically refers to the population (that is all instances of $B$)
  • $b$ is a specific draw from $B$, it refers to a sample (in contrast to the population)
  • $\beta$ is a parameter. It is not directly measured. For instance, the slope in a particular relationship/function/model.
  • $\hat \beta$ is an estimate for $\beta$ based on a sample.
$\endgroup$
  • $\begingroup$ I am trying to understand the reply but there is a b, B, italic B, and beta in the answer. Should I continue to try to understand the reply, or is there room for updating the notation? $\endgroup$ – Snoopy Feb 9 '18 at 15:39
  • $\begingroup$ I will place my comment in the answer since these conventions are also troubling to me so maybe someone can help. $\endgroup$ – Martijn Weterings Feb 9 '18 at 17:21
  • $\begingroup$ As you can see I like to use lowercase and uppercase differently. You used lowercase for an estimate of uppercase. Whereas I use lowercase as a particular draw from uppercase. $\endgroup$ – Martijn Weterings Feb 9 '18 at 17:29
  • $\begingroup$ Isn't $\hat\beta$ an estimator for $\beta$, and estimate is $b$? $\endgroup$ – Parthiban Rajendran Nov 20 '18 at 3:46
  • $\begingroup$ $\hat\beta = (X^TX)^{-1}X y$ is a statistic. It is based on a particular transformation of $y$ that is sampled from the distribution $Y$ (if you like you could call it 'an estimator for $\beta$'). Like for any sample that comes from a distribution with finite mean, you have that a single sample $\hat\beta$ is an unbiased estimator for the mean/expectation-value of it's own distribution $E(\hat\beta)$, this is not something particular for OLS. $\endgroup$ – Martijn Weterings Nov 20 '18 at 9:02
0
$\begingroup$

Following your notation and your particular example ... "given certain assumption we have that"

E[b | X] = B

for the law of iterated exeptaction it follows that:

E[b] = B

$b$ is your ESTIMATOR for $B$ and it is a random variable because its value depends on the sample you draw. It's centered around the $B$ that is its expected value (the mean). Every time you draw a sample the value o $b$ is different but the mean of $b$, $E(b)$ is equal to $B$ .

The mean of the sampling distribution of $b$ is just $B$ as you can read in the equality above.

$\endgroup$
  • $\begingroup$ But my question is not about what b is or what its expected value is. My question is about "Est. E[b | X]". It is like "Est. Var[b | X]" which is s^2(X'X)^{-1}. $\endgroup$ – Snoopy Feb 9 '18 at 20:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.