I am working with a dataset that includes variables $Y$ and $X$. I assume that

$$ Y = \beta X + \epsilon $$

satisfies all the assumptions of OLS. Based on industry knowledge, I know that theoretically $\beta = 1$. However, I am aware that $X$ contains noise, meaning I have observed $\hat{X} = X + u$ where $u$ represents the noise. When I run the OLS estimate, I find that the estimated $\beta$ is less than 1 due to the noise in $X$, which inflates the variance of $X$.

I have different pairs of $(Y, X)$ from various groups. I can verify my assumption by observing that in larger groups (where data variability is reduced by the Central Limit Theorem), the fitted $\beta$ is closer to 1.

Given this, my question is: If I want to make an out-of-sample forecast, should I use 1 as my $\beta$ or should I use the OLS $\beta$ estimate, which is biased?

On one hand, it seems that using the unbiased $\beta$ of 1 is preferable since it is theoretically unbiased. However, if I assume the same noise $u$ is present in my out-of-sample $X$, my residual term will have larger variance because it includes the term $\beta^2 \text{Var}(u)$. Using a smaller, biased $\beta$ might help reduce the residual variance, despite the bias.

Is this a bias-variance trade-off scenario? What is the best approach here if my goal is to minimize the out-of-sample residual MSE?

  • $\begingroup$ where data variability is reduced by the Central Limit Theorem What does this mean? $//$ How do you know that $\beta = 1?$ What about an intercept term? $//$ should I use the OLS β estimate, which is biased? Why would the OLS estimator be biased? $\endgroup$
    – Dave
    Commented Jul 10 at 21:59
  • $\begingroup$ OLS is biased because it doesn’t assume noise in X. So if you do ols estimate with cov(x_observed, y)/var(x_observed), it will become cov(x,y)/(var(x)+var(u)), assuming u is independent from y. So basically your beta coefficient will be shrinked. For the data I can’t disclose too much since it is confidential. But it is some physical data and we know that beta should be 1 in theory $\endgroup$
    – The One
    Commented Jul 10 at 22:27
  • $\begingroup$ Then it sounds like you have a good starting point to craft an informative prior distribution and approach this with Bayesian estimation. $\endgroup$
    – Dave
    Commented Jul 11 at 3:06


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.