Consider the linear regression problem $$(A+\Delta A)x = b + \Delta b.$$

If $\Delta A = 0$ and $\Delta b$ is identically and independently distributed, then ordinary least-squares gives a good (BLUE) estimate for $x$. If $\Delta A = 0$ and $\Delta b$ has a given general covariance, then generalized least-squares gives a good estimate (BLUE). If there is error in both $A$ and $b$ such that $vec[\Delta A, \Delta b]$ are identically and independently distributed, then total least-squares give a good (strongly consistent) estimator of $x$.

What are good/efficient estimators in the general case that $vec[\Delta A, \Delta b] \sim \mathcal{N}(0,\Sigma)$?

  • 1
    $\begingroup$ If $b$ is iid then necessarily $A$ is a single column of constant values. It looks like you are consistently confusing assumptions about $b-Ax$ with assumptions about $b,$ but because of that your question is unclear. Also, under what conceivable circumstances would the entries of $A$ and $b$ be iid?? $\endgroup$ – whuber Mar 26 at 11:28
  • 2
    $\begingroup$ Yes, thank you for helping clarifying the issue! I was confusing assumptions on the errors with assumptions on the data themselves (facepalm). I have edited the question. The errors $\Delta A$ and $\Delta b$ can be iid, for example, if they where generated by the same process (which now makes sense?). $\endgroup$ – fhchl Mar 26 at 12:04

There exists a paper by van Huffel about the generalized total least squares estimator, where the entries of $A$ can be correlated.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.