Estimating number of compliers Be $X$ a dichotomous endogenous variable and $Z$ its dichotomous instrumental variable. Suppose that for compliers if $Z_i=0$ then $X_i=0$ and if $Z_i=1$ then $X_i=1$. 

Assuming that defiers do not exist, we can easily estimate the number of compliers from the sample by the formula:
(number of individuals with $Z_i=0$ and $X_i=0$)-(number of individuals with $Z_i=1$ and $X_i=0$)+(number of individuals with $Z_i=1$ and $X_i=1$)-(number of individuals with $Z_i=0$ and $X_i=0$)
But how is it possible to estimate the number of compliers if the endogenous variable $X$ is not dichotomous? And what if both the endogenous variable $X$ and the instrumental variable $Z$
are not dichotomous?              
 A: Binary treatment, binary instrument
For the binary treatment and instrument the size of the complier group is given by the Wald first-stage
$$P[X_{i1} > X_{i0}] = P[X_{i}|Z_i=1] - P[X_{i}|Z_i=0]$$
If you take this first stage, multiply it by the probability that the instrument is switched on $P[Z_i=1]$ and divide by the number of all treated $P[D_i=1]$ you will get the proportion of compliers among the treated individuals.
Multivariate treatment, binary instrument
With a binary instrument and a multivariate treatment you can learn about the size of the compliers using the average causal response (ACR) theorem. Suppose $X_i$ takes values $ \{ 0,1,...,M \} $, then the relative size of compliers at each point $x$ in the support of $X$ is:$$P[x_{i1}\geq x>x_{i0}] = P[x_{i1}\geq x] - P[x_{i0} \geq x]$$
which you would need to calculate at every value of $X$. These are the people who are driven from a treatment intensity strictly below $x$ to a treatment intensity of at least $x$. However, this gives the actual size of the compliant subpopulation if and only if every individual moves up the treatment intensity by one unit and not more. If a complier goes from $x_{i0} = 2$ to $x_{i1} = 5$ then she will be counted in each of the subpopulations
$$
\begin{align}
x_{i0} &< 3 \leq x_{i1} \newline
x_{i0} &< 4 \leq x_{i1} \newline
x_{i0} &< 5 \leq x_{i1} \newline
\end{align}
$$
If you are interested in the size of the compliant subpopulation you would need to adjust for this double/multiple counting of individuals that switch treatment intensity by more than one unit. As a side note: for the LATE estimator this is a good thing because it will weight those observations more which are affected the most by the instrument.
Binary treatment, multivariate instrument
Suppose your instrument $Z_i$ takes on values $\{ 0,1,...,J \}$. The compliant subpopulation for each value of the instrument is
$$P[X_i=1|Z_i=j] - P[X_i=1|Z_i=j-1] $$
at every value that the instrument takes. The issue regarding the actual amount of compliers is the same as before. The count is correct if every treated subject was subjected to a one unit increase in the instrument only. An individual switching from instrument value $j-2$ to $j$ is counted twice.
Multivariate treatment, multivariate instrument - no answer
Probably the most interesting case for you but I have to disappoint you here as I have no answer to this part. When both the instrument and the treatment are multivariate the problem of counting compliers becomes even more delicate as the dimension increases in two directions. This worsens the issue of multiple counts and personally I wouldn't know how to count compliers in this case. Maybe somebody is more versed in this topic and can help you out on the last part.
If you want more detail on the previous topics have a look at Angrist and Pischke (2009) "Mostly Harmless Econometrics" who provide a good discussion and the derivations (at least for the first two headings).
