Is there any other unbiased estimator of regression coefficient than OLS? For instance, one might consider using unbiased estimator with less computational cost (since OLS involves matrix inversion)?

  • 2
    $\begingroup$ This is an interesting question! I take it we can assume that the sample size ($n$) is larger than the number of features ($p$)? Also, just to stress: matrix inversion is not required to find the OLS estimates. Only solving a linear system is required $\endgroup$
    – Ben
    May 13 at 0:47
  • $\begingroup$ Special case 1: If the design matrix $X \in \mathbb{R}^{n \times p}$ is orthogonal, then OLS can be calculated in $2np$ flops, which will is hard to beat. Special case 2: if $X$ is the identity, the response vector itself is an unbiased estimator, which can be computed in $0$ flops. Perhaps there's more gains for less structured design matrices, though $\endgroup$
    – Ben
    May 13 at 0:53
  • $\begingroup$ The title does not seem to quite match the question; consider updating. $\endgroup$ May 13 at 9:47


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.