The inverse (right inverse in discrete case) of the cumulative distribution function $F(x)$ is called the quantile function, often denoted $Q(p)=F^{-1}(p)$. The expectation $\mu$ can be given in terms of the quantile function (when the expectation exists ...) as
$$
\mu=\int_0^1 Q(p)\; dp
$$
For the continuous case, this can be showed via a simple substitution in the integral: Write
$$
\mu = \int x f(x) \; dx
$$ and then $p=F(x)$ via implicit differentiation leads to $dp = f(x) \; dx$:
$$
\mu = \int x \; dp = \int_0^1 Q(p) \; dp
$$
We got $x=Q(p)$ from $p=F(x)$ by applying $Q$ on both sides.
For the general case, we can interpret $E(X) = \int x dF_X(x)$ as a Riemann–Stieltjes integral which, when the range of $X$ is the finite interval $[a,b]$, is defined as a limit of approximating sums of the form, for partitions $a=x_0<x_1<\dotsm<x_n=b$,
$$\sum_{i=0}^{n-1} x_i^* \left[F(x_{i+1})-F(x_i)\right] $$ Passing to the quantile function $F^*$ (read the $^*$ as indicating a generalized inverse), and using $p_i=F(x_i)$, $p_i^*=F(x_i^*)$ this goes over to the approximating sum
$$ \sum_{i=0}^{n-1} F^*(p_i^*) \left[ p_{i+1} - p_i \right] $$
which are approximating sums for
$$ \int_0^1 F^*(p) \; dp $$
Then the infinite range case is treated the same way as with Riemann integrals.