in my statistics course, it was mentioned that for the Ordinary least squares method (OLS) taking variables $X_i$ and $Y_i$ as the response variable, the random variable is solely $Y_i$.

However for the Orthogonal regression method (ORM) both the $X_i$ and $Y_i$ are considered to be random variables.

From my understanding in the OLS we optimize the regression based on individually picked values, so for each $x_i$. However, in the ORM the optimization is not made for some fixed $x_i$ value. Is this the correct reason, or is there something else?


1 Answer 1


It's not true that, with OLS, only the dependent variable, $Y$, is random. In fact, both can be random variables, however OLS is centered on minimizing the mean squared deviation between $Y$ and $X\beta$, conditioned on the variable $X$. That is, if your equation is $$Y = X\beta + e$$ where $e$ is a vector of IID components, then the least squares problem is to find the vector $\beta$ such that $$\mathbb{E}[(Y- X\beta)^2\mid X]$$ is minimized.

In this sense, we can often "treat" the random variable $X$ as being "non-random", since we can pull it out of the operator $\mathbb{E}[ \cdot \mid X]$, however it still can be a random variable.

ORM has to do with dealing with cases where both $X$ and $Y$ contain measurment error. That is, when $X = X + \epsilon$ and $Y = y + \mu$ where $x,y$ are unknown fixed constants and $\epsilon, \mu$ are independent random variables.

  • 1
    $\begingroup$ Alright!, I understand now why the prof mentioned that the ORM can be more appropriate when there is uncertainties with both variables. $\endgroup$ Commented Oct 9, 2016 at 0:13
  • $\begingroup$ My contribution is to add a little correction on Digio contribution. First of all, thank for your answer, it is very clear. It is not a particular comment but the fact that it is not $ X $ we are going to find but the vector $\beta$ who minimize the quantity $$ \mathbb{E} [ (Y - X \beta )^ 2 | X ] $$ $\endgroup$ Commented Dec 16, 2023 at 3:41
  • $\begingroup$ @ThegoatKonan Thank you for noticing that. I have fixed this evident typographical error by changing "$X$" to "$\beta.$" $\endgroup$
    – whuber
    Commented Dec 16, 2023 at 17:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.