You can quantify the quality of the estimator by calculating the total surprisal of all of the coin flips.

Suppose that your expert makes predictions $q_i$ for each coin.  Then, given indicator variables for the coins coming up heads $x_i$, the total surprisal is:

\begin{align}
\sum_i\left[ -x_i\log q_i - (1-x_i)\log (1-q_i)\right].
\end{align}

The expected value of the surprisal given the true values $\{p_i\}$ is the cross-entropy:
\begin{align}
\sum_i \left[-p_i\log q_i  -(1-p_i)\log (1-q_i)\right].
\end{align}
It is nonnegative, and achieves its minimum value (the entropy of $\\{p_i\\}$) if and only if $p_i = q_i \forall i$.

If you subtract the entropy from the cross-entropy, you get the relative entropy (whose minimum value is zero).  If you take $e^{-x}$ of that, you have a number in $[0, 1]$ as you wanted with a reasonable probabilistic interpretation.