3
$\begingroup$

It is an interview question, I simplify the question as dim = 1, it is better if the solution is about multiple dimension.

We know that the assumption of OLS is

$$y = ax+b+\epsilon,\quad \epsilon\sim N(0,\sigma^2).$$

And its estimation is unbiased. Then if we change the assumption to

$$y = a(x+\epsilon)+b,\quad \epsilon\sim N(0,\sigma^2),$$

what's the bias of estimation?

It seems we should solve the estimation of $a,b$ by MLE first (replace $\sigma$ by $a\sigma$) and the solution is no longer equivalent to OLS solution, which is more complicate.

$\endgroup$
1
  • 1
    $\begingroup$ How far do you get when you try to take the expectation of the coefficient estimates? $\endgroup$
    – Taylor
    Oct 17, 2021 at 19:19

1 Answer 1

2
$\begingroup$

$$y = a(x+\epsilon)+b,\quad \epsilon\sim N(0,\sigma^2)$$

is the same as

$$ y= ax+b+a\epsilon,\quad \epsilon\sim N(0,\sigma^2)$$

and

$$ y= ax+b+\epsilon,\quad \epsilon\sim N(0,a^2\sigma^2)$$

So it is the same situation, except the error terms have different variance.


This only remains the same for simple linear regression. If there are more terms, e.g quadratic terms then there will be bias.


Another thing is when the measurements of $x$ have an error. So the $x$ that we use in the estimate, is not the real $x$. In that case you have an errors in variables model and the OLS estimator will be biased.

https://en.m.wikipedia.org/wiki/Errors-in-variables_models

$\endgroup$
7
  • $\begingroup$ So it is the same situation, except the error terms have different variance I don't understand that how you know the bias of estimation without solving the estimation (get the representation of estimation)? And the solving process is totally different with OLS as $a\sigma$ occurs in MLE. $\endgroup$ Oct 18, 2021 at 6:39
  • $\begingroup$ @user6703592 the $a$ in $a\sigma$ is just a constant term that scales the noise. When you speak about OLS then it is about estimating $a$ and $b$ and nothing changes. $\endgroup$ Oct 18, 2021 at 7:11
  • $\begingroup$ @user6703592 I suspect that the interview question might have been about the errors-in-variables situation that I mention at the end of the answer. It is different from what you wrote down. But you say that you 'simplified' the question. Maybe you also changed the point of the interview question by doing this? $\endgroup$ Oct 18, 2021 at 7:13
  • $\begingroup$ For the point of EIV, I agree with you. But for maximizing MLE subject to the variable $a,b$ under my statement above, I cannot agree with you. The objective function has been changed, the solution of estimations of $a,b$ must be different. Since $y \sim N(ax+b, a^2\sigma^2),$ where both mean and variance have variables. $\endgroup$ Oct 18, 2021 at 7:17
  • $\begingroup$ @use46703592 I see what you mean now. If $\sigma$ is known (is that condition part of the question?) then the magnitude of the residuals also gives information about the coefficient $a$. But if $\sigma$ is unknown then using MLE is not different. The situation for OLS is not giving a biased result in any case (does the interview question suggest which estimation method to use?). $\endgroup$ Oct 18, 2021 at 7:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.