1
$\begingroup$

In a simple regression model using OLS, why is it not, at least in general, possible to move the $x_i$'s to the left-hand side of the model and the $y_i$'s to the right hand side when I want to switch the explanatory and response variable? Intuitively, it seems fairly clear that if ${\hat \beta ^{OLS}}\left( {Y,X} \right) = \mathop {\arg \min }\limits_{\beta \in \mathbb{R}} \sum\limits_{i = 1}^n {{{\left( {{y_i} - x_i^T\beta } \right)}^2}} $ and ${\hat \gamma ^{OLS}}\left( {X,Y} \right) = \mathop {\arg \min }\limits_{\gamma \in \mathbb{R}} \sum\limits_{i = 1}^n {{{\left( {{x_i} - y_i^T\gamma } \right)}^2}} $, the simple "switching of sides" would not work since I'm minimizing two different sums. But is there any more formal explanation why it cannot be done?

$\endgroup$
  • $\begingroup$ How more formal than you have already wrote? $\endgroup$ – Alecos Papadopoulos Jun 14 '14 at 18:00
1
$\begingroup$

Not sure if these fit within your notion of formal here, but:

  1. The meaning of your errors and residuals changes, since they are defined by deviation from the left hand side of the model.

  2. After the switch, if there are multiple $x$s on the left side, this would imply a possibly correlated errors (multivariate) model, or a mis-specified model if you start from least squares multiple regression, and do not move to a multivariate model.

  3. If your data are derived from a study design that supports causal inference, then you may be violating causality by reversing causation (e.g. heart attacks causing people to have a smoking history). This can extend to cross-level fallacies depending on the kind of inference you make out of your results (e.g. if $x$s are compositional population-level variables and $y$ is an individual-level variable, you may be asserting that individual experiences are creating (non-compositional) population-level contexts).

$\endgroup$
  • $\begingroup$ Thank you, I think this response should be formal enough, also the reversing causality is a thing I did not realize before, or at least not to such extent. $\endgroup$ – Skumin Jun 15 '14 at 7:59
  • $\begingroup$ Skumin - When you say "I think this response should be formal enough" --- formal enough for what exactly? What thing do you need a formal response to? $\endgroup$ – Glen_b Jun 16 '14 at 0:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.