1
$\begingroup$

Suppose I fit a linear model where Y ~ X1 + X2 + error. This model performs quite well. Now suppose I want to reverse this and estimate X1 and X2 given that I have observed Y.

I'm curious what is the most appropriate way to do this? Is it fitting two regressions where one of the variables (either X1 or X2) is latent? Is it some sort of parameter grid search where I have X1 ~ Y + X2 for a fixed X2 and I do a grid search over X2? (and then I reverse it and do the same for X1)?

Thank you for your suggestions!

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Let's consider a simpler problem. Imagine we had no noise at all, so we had an exact equation like $y=a + b x_1 + c x_2$. For any given value of $y$, say $y=10$, there's an infinite number of possible pairs of ($x_1,x_2)$ solutions that lie along a line (the one that's formed by the intersection of the two planes $y=a + b x_1 + c x_2$ and $y=10$). If you specify one, you can work out the other: $10=a + b x_1 + c x_2 \implies x_2=(10-a-b x_1)/c$. Is the line $a-10 + b x_1 + c x_2 =0$ the kind of "solution" you're after here? $\endgroup$
    – Glen_b
    Commented Jan 14, 2018 at 2:45
  • 1
    $\begingroup$ @Glen_b: Yes, that would work. So the thought I had was to fix one and then find the solution of the other. So what i could do is fix one over a grid, do a regression over the remaining variable. I can then repeat the regression multiple times and choose the pair that gives me the best R^2 etc. $\endgroup$
    – user1357015
    Commented Jan 14, 2018 at 4:00
  • $\begingroup$ Your question relates to inverse regression, but with multiple predictors. $\endgroup$
    – Glen_b
    Commented Jan 14, 2018 at 8:43

1 Answer 1

Reset to default
1
$\begingroup$

In the generality that you described, this is a problem about estimating the joint distribution of X1, X2 and Y. Depending on the data size, you can do this either parametrically (medium-to-large sample size) or nonparametrically (large sample size). All conditional means will come as an easy follow-up calculation. You may notice that they are not linear. So linear regression may or may not be suitable here.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.