3
$\begingroup$

Consider a linear regression model. Suppose that the estimator $\hat{\beta}$ for the vector of the parameters of the model $\beta$ is, for some reasons, biased. As a consequence:

$$E[\hat{\beta}] = \beta + k$$

where $k$ is the unknown bias (positive or negative).

The question is: what happens to the t-test and to the F-test? Do we reject the null hypothesis more often? Do we reject it less often? Can we trust the p-values?


As far as I know, the t-test is calculated in the following way:

$$\frac{\hat{\beta}-0}{\sqrt{Var(\hat{\beta})}}\tag{1}$$

But since the estimator $\hat{\beta}$ is biased, if we knew the bias $k$, I'd calculate the test in this way:
$$\frac{\hat{\beta}-k-0}{\sqrt{Var(\hat{\beta})+k^2}}\tag{2}$$

(The standard error of an estimator is the square root of the MSE and the MSE can be written as the sum of the variance of the estimator and the squared bias of the estimator.)

So it seems that, due to the bias, using the test statistic $(1)$ instead of $(2)$, the values of the test statistic are higher than they should be, since the denominator is lower than it should be. Is it correct? Does it matter if the bias is positive or negative?

$\endgroup$
  • $\begingroup$ You might want to look into the non-central t-distribution. You are correct in that the t-statistic would not be reliable. $\endgroup$ – JohnK Jan 11 '16 at 16:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.