If X can predict Y in regression, why isn't Y guaranteed to predict X? Intuitively, if X can predict Y in a multiple linear regression model,

$y$ = $\beta_0$ + $\beta_1$$X$ + $\beta_2$$Z$ + $e$

X and Y are associated. Since there's an association between them, why is it not guaranteed that I could predict X by Y? Why is the relationship asymmetric? Or is my conception incorrect?
 A: When you use the regression equation to make a prediction by plugging in a value of $x$, you are not predicting the value of $y$ for that value of $x$. You are predicting the mean of the $y$-values for that value of $x$. In detail:
The regression equation
$$y = \beta_0 + \beta_1 x + \epsilon$$
says that $y$ is equal to a linear function of $x$ plus some random scatter. If you set $x=3$, say, you have
$$y = \beta_0 + 3\beta_1 + \epsilon$$
and there is still some random scatter there. In other words, you are saying that "my prediction is that $y$ is normally distributed with mean equal to $\beta_0 + 3\beta_1$". To get an actual value for $y$, you need to take the expectation. So you are saying that "The mean of all the $y$-values for which $x=3$ is $\beta_0 + 3\beta_1$".
If you make a prediction by inverting the regression equation, say by plugging in $y=4$, then you are saying "The $x$-value for which the mean of all the corresponding $y$-values is equal to $4$ is $(4-\beta_0)/\beta_1$", which isn't usually the kind of prediction that you want.
Statistical courses often don't help by talking about "the line of best fit", which makes it sound like the situation is symmetrical in $x$ and $y$, which is very not the case. Recently there was a debate on the ANZSTAT mailing list, and someone posted a link to a good introductory course which explains it well:
https://www.stat.berkeley.edu/~stark/SticiGui/Text/regression.htm
A: The answer depends on what you mean by “predict”. If you imply any kind of causation then obviously it is one way road. Suppose sunrise causes you to wake up. If I wake you up in the middle of the night sun will not rise suddenly.
On the other hand, if you mean by predicting an explanatory power of X in a multiple regression Y~1+X+Z or explanatory power of Y in a regression X~1+Y+Z, then it is a different story. You can certainly invert the relationship algebraically as long as it is strong. If I know what time you wake up I can predict when the sun rises.
I conditioned on relationship being "strong" because when you invert the equation your optimization problem changes. Instead of minimizing squares $(\hat y_i-y_i)^2$ we minimize  squares $(\hat x_i-x_i)^2$. It is a different equation that may not render a significant relationship if the relationship was weak to start with. For instance, you can find that $\beta_1$ is significant, i.e. X predicts Y in this narrow definition. Yet, when you invert the equation you may end up with a model where coefficient of Y is not significant, so in this sense Y does not predict X.
A: This is an interesting question. In the case of single variable linear regression, there is a (assumption) symmetric relationship but the same is not true for multiple linear regression.
A symmetric relationship can exist for some problems, a contrived example is training a linear regression model to behave like an AND gate. 
If you ask why is this not "guaranteed" then there is a clear counterexample (proof by construction) which is training an OR gate using linear regression. You can learn a model that can predict the output of the gate $Y$ given 2 inputs $X_1, X_2$ accurately but the reverse is not possible.
Another way to think about it is that the multivariable linear regression model learns a many-to-one dimension mapping. And since many different points on the input can map to the same point on the output the reverse mapping is ambiguous.
UPDATE: The other answers seems to explain why the linear-regression objective (or loss function) is not symmetric. But I think you are asking a different question which is (paraphrasing), once I learn a linear relationship between $Y$ and $X_1, X_2$, why can't I use it in reverse? 
