5
$\begingroup$

in my statistics course, it was mentioned that for the Ordinary least squares method (OLS) taking variables $X_i$ and $Y_i$ as the response variable, the random variable is solely $Y_i$.

However for the Orthogonal regression method (ORM) both the $X_i$ and $Y_i$ are considered to be random variables.

From my understanding in the OLS we optimize the regression based on individually picked values, so for each $x_i$. However, in the ORM the optimization is not made for some fixed $x_i$ value. Is this the correct reason, or is there something else?

$\endgroup$
3
$\begingroup$

It's not true that, with OLS, only the dependent variable, $Y$, is random. In fact, both can be random variables, however OLS is centered on minimizing the mean squared deviation between $Y$ and $X\beta$, conditioned on the variable $X$. That is, if your equation is $$Y = X\beta + e$$ where $e$ is a vector of IID components, then the least squares problem is to find the vector $X$ such that $$\mathbb{E}[(Y- X\beta)^2\mid X]$$ is minimized.

In this sense, we can often "treat" the random variable $X$ as being "non-random", since we can pull it out of the operator $\mathbb{E}[ \cdot \mid X]$, however it still can be a random variable.

ORM has to do with dealing with cases where both $X$ and $Y$ contain measurment error. That is, when $X = X + \epsilon$ and $Y = y + \mu$ where $x,y$ are unknown fixed constants and $\epsilon, \mu$ are independent random variables.

$\endgroup$
1
  • 1
    $\begingroup$ Alright!, I understand now why the prof mentioned that the ORM can be more appropriate when there is uncertainties with both variables. $\endgroup$ Oct 9 '16 at 0:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.