How to quantify the asymmetry of a probabilistic dependency? Let $X$ and $Y$ be dependent random variables. Is there a typical way to quantify how much more/less knowing $X$ tells you about $Y$ than knowing $Y$ tells you about $X$?
For example, if $Y=X^2$, then $X$ completely determines $Y$ whereas $Y$ can only reduce the possibilities of $X$ down to two.
\begin{align}
p(Y|X{=}x)\ &=\ \delta(Y-x^2)\\[6pt]
p(X|Y{=}y)\ &=\ \frac{1}{2}\delta(X-\sqrt{y})+\frac{1}{2}\delta(X+\sqrt{y})
\end{align}
Perhaps a more tangible example is if $X$ is your {latitude,longitude} coordinates and $Y$ is the country you're in. If you tell me your {latitude,longitude} then I precisely know what country you're in, but if you tell me what country you're in then my knowledge of your {latitude,longitude} improves, but not "as much."
My first thought is to compare a dissimilarity metric $D$ (perhaps KL-divergence) between the marginals and conditionals. Something like,
$$
\text{"dependence_asymmetry"} = \frac{D\big{[}p(Y) \parallel p(Y|X)\big{]}}{D\big{[}p(X) \parallel p(X|Y)\big{]}}
$$
Then I looked at conditional entropy and thought perhaps it would make sense to use,
$$
\text{"dependence_asymmetry"} = H_{Y|X} - H_{X|Y}
$$
Interestingly, that quantity is equal to $H_Y - H_X$, which seems to make sense with my examples.
I understand that in the limit of independence, there can be no asymmetry.
$$
p(Y|X)=p(Y)\ \iff\ p(X|Y)=p(X)
$$
But it seems possible and interesting when $p(Y|X) \approx p(Y)$ while $p(X|Y)$ differs greatly from $p(X)$ in some way.
In summary, how should one quantify this asymmetry of a probabilistic dependency? Or is this concept fundamentally flawed?
 A: I think this is quite an interesting idea, and worth pursuing.  For the most part, we tend to formulate statistical and probabilistic quantities based on what we want them to do, so you will also need to give some thought to what you want to use this quantity for.  The purpose will assist you to determine what kinds of properties you want the quantity to have.  In the present case, where there is no particular application in mind, things are a bit more open-ended, but it might be possible to make some progress by specifying some properties you want for your measure and then formulating something intuitive that meets the required properties.
To get you started, I'm going to suggest some desireable properties for a measure that encapsulates what you want here.  The first thing you need to decide is how you want to quantify the "dependence asymmetry" at issue --- you appear to have chosen to try to quantify this by a real number.  Following this approach, suppose we let $\mathbb{D}: (X,Y) \mapsto \mathbb{R}$ denote your measure for "dependence asymmetry", where the measure gives a real value, with positive values representing asymmetry in favour of $X$ (i.e., $X$ gives more information about $Y$ than $Y$ gives about $X$) and negative values representing asymmetry in favour of $Y$.  I would recommend that your measure should posess the following properties:
$$\begin{align}
\text{Reflexivity} & & & & & & & \mathbb{D}(X,X) = 0, \\[6pt]
\text{Anti-symmetry} & & & & & & & \mathbb{D}(X,Y) = -\mathbb{D}(Y,X), \\[6pt]
\text{Dependence Symmetry} & & & & & & & F_{X,Y}=F_{Y,X} \implies \mathbb{D}(X,Y) = 0, \\[6pt]
\text{Monotonicity} & & & & & & & \mathbb{D}(f(X),Y) \leqslant \mathbb{D}(X,Y), \\[6pt]
\text{Injection Independence} & & & & & & & f \text{ is injective } \implies \mathbb{D}(f(X),Y) = \mathbb{D}(X,Y). \\[6pt]
\end{align}$$
The reflexivity and symmetry properties are fairly natural for this type of measure.  Reflexivity reflects the fact that any random variable gives the same amount of information about itself as itself gives about it (i.e., full information).  Anti-symmetry is a useful property to impose to ensure that your measure is not fundamentally dependent on the order in which the random variables enter the measure (other than by a change of sign).  Dependence symmetry says that exchangeability of the joint distribution of the underlying random variables implies dependence symmetry, which is a reasonably condition, since exchangability is symmetry of the entire joint distribution.  The monotonicity property represents the fact that $f(X)$ cannot give more information than $X$ for any function $f$ and the injection independence condition says that an injective function of a random variable does not change the dependence symmetry.

A starting point for a proposed measure: Since you are trying to encapsulate the level of uncertainty about one random variable given knowledge of the other (specifically the asymmetry in this), a natural approach is to contruct an asymmetry measure using the conditional variances $\mathbb{V}(X|Y)$ and $\mathbb{V}(Y|X)$.  One possible starting point for a measure for this concept which might be reasonable is:
$$\mathbb{D}(X,Y) = \mathbb{E} \bigg[ \log \mathbb{V}(X|Y) - \log \mathbb{V}(X) - \log \mathbb{V}(Y|X) + \log \mathbb{V}(Y) \bigg].$$
This measure takes the underlying difference in the conditional log-variances (which gives a random variable depending on both $X$ and $Y$) and it takes the expected value of this quantity.  This measure obeys reflexivity, anti-symmetry and dependence symmetry, but it doesn't yet obey monotonicity or injection independence; the reason for this is that the variance is not invariant to functions applied to the underlying random variables.  I would suggest thinking about how you could adjust this measure, or formulate a different measure, which obeys all the properties.
