What is the variation of information of a variable with its self? Variation of information measures the uncertainty we expect in one variable if we are told the value of another variable. It is computed as
$$VI(X,Y) = H(X) + H(Y) - 2 I(X,Y)$$
or
$$VI(X,Y) = H(X,Y) - I(X,Y)$$
where $H()$ is entropy and $I()$ is mutual information.
Mutual information, when computed for a single random variable with itself ($I(X,X)$), just gives the entropy of that variable,
$$I(X,X) = H(X)$$
If I similarly calculate variation of information for a single random variable with itself, what is the resulting answer and interpretation?
$$VI(X,X) = H(X,X) - I(X,X) = H(X,X) - H(X) = ?$$
and why does it come out as infinity?
 A: It should come out to be 0, because $VI$ is a metric, which requires that $VI(X, X) = 0$. A more intuitive way of understanding this is to use the equation
$$
VI(X, Y) = H(X | Y) + H(Y | X)
$$
(see the Wikipedia page) instead. Here, $H(X | Y)$ is the conditional entropy of $X$ given $Y$, which has a nice interpretation as the amount of information needed to describe the outcome of the variable $X$ when given that the value of the random variable $Y$ is known. In this case, if we already know the value of $X$, then we don't need any additional information to describe the outcome of $X$, so intuitively we should have $H(X | X) = 0$. Hence, $VI(X, X) = H(X|X) + H(X|X) = 0$.
If we want to be more rigorous, we can calculate $H(X | X)$ using the definition:
$$
H(X | Y) = \int p(x, y) \log ( p(x | y) ) dxdy
$$
where $p(x, y)$ is the joint density of $X$ and $Y$, and $p(x | y)$ is the conditional density of $X$ given $Y$. In the case where $X = Y$, the joint density is singular because the support is $A = \{ (x, y) \in \mathbb{R}^2 : x = y \}$, which has Lebesgue measure 0 in $\mathbb{R}^2$. The following proof is hard to follow with the standard abuse of notation of using $p(\cdot)$ to mean the density of different variables depending on different arguments, so let's use $f$ to denote the density of $X$ and $g$ to denote the joint (singular) density of $X$ and $X$. We can still compute the integral despite this by using a simple parametrization:
\begin{align}
H(X | X) & = \int_{\mathbb{R}^2} p(x, y) \log (p(x | y)) dx dy \\
& = \int_A g(x, y) \log \left( \frac{g(x , y)}{f(y)} \right) dx dy \\
& = \int_\mathbb{R} g(t, t) \log \left( \frac{g(t , t)}{f(t)} \right) dt \text{ by parametrizing $A$ using $z(t) = (t, t)$}\\
& = \int_\mathbb{R} f(t) \log \left( \frac{f(t)}{f(t)} \right) dt\\
& = \int_\mathbb{R} f(t) \log \left( 1 \right) dt\\
& = \int_\mathbb{R} f(t) (0) dt\\
& = \int_\mathbb{R} 0 dt\\
& = 0
\end{align}
If you want to use the equation
$$
VI(X, X) = H(X, X) - H(X)
$$
we can also rigorously prove that $H(X, X) = H(X)$, and thus this can be seen to be 0 as well. Similarly to above, we have
\begin{align}
H(X, X) & = \int_{\mathbb{R}^2} p(x, y) \log ( p(x, y) ) dxdy \\
& = \int_A g(x, y) \log ( g(x, y) ) dxdy \\
& = \int_\mathbb{R} g(t, t) \log ( g(t, t) ) dt \text{ by the same parametrization as before} \\
& = \int_\mathbb{R} f(t) \log ( f(t) ) dt \\
& = \int_\mathbb{R} p(x) \log ( p(x) ) dx \\
& = H(X)
\end{align}
