McNemar's Test
I don't think you can apply McNemar's test on the vector of precisions. Instead, you would apply it to the 2x2 contigency table of cross-classification of test items:
$$
\begin{array}{lll}
& \mbox{Correct in C1} & \mbox{Incorrect in C1} \\
\mbox{Correct in C2} & m_{11} & m_{12} \\
\mbox{Incorrect in C2} & m_{21} & m_{22} \\
\end{array}
$$
formed over all 30 classes (which would give you the average over classes, weighted by their incidence in the test data). Edit: This table is formed using the performance each classifier has on each particular item, and assumes that the same items are shown to each classifier. So the vectors of precisions $\mathbf v_1$,$\mathbf v_2$ will not suffice.
McNemar's test is then testing that the probability of C1 being incorrect while C2 is correct ($p_{12}$) equals the probability of C1 being correct while C2 is incorrect ($p_{21}$). See wikipedia.
Comment on McNemar's Test
McNemar's test will undoubtedly be more sensitive than using the precision vectors, because it is controlling for the variability due to the underlying difficulty of each item being classified. If you reject then you know that the misclassification rates differ by classifier, averaging over the background prevalence of the 30 categories. This is testing something a little bit different than the precisions, but seems pretty reasonable as long as each classifier must return a label from one of the 30 categories (ie, it can't refuse to classify an item).
Alternatives not requiring item-level cross tabs
If you don't have the 2x2 cross-classification table, then you would indeed be forced to do something with your vector of precisions. A couple issues here:
- Your vectors $\mathbf{v}_1$ $\mathbf{v}_2$ probably have different variances, because each component $v_{ij}$ is an average over a varying number of items, so will have variances proportional to $n_{ij}^{-1/2}$, where $n_{ij}$ is the number of detected positives for class $i$, classifier $j$. So unless you account for this, you will not have correct inference (a test will not have the correct rejection rate).
- As binomial proportions, the variances of $\mathbf{v}_1$ $\mathbf{v}_2$ also depend on the usual fashion on $v_{ij}(1-v_{ij})$.
1 and 2 suggest that if you indeed cannot recover the pairing between the classifiers, that you should use binomial regression on the model:
$$E ( Y_{ij} | n_{ij} ) = \alpha + \beta x_j$$
where $Y_{ij}, n_{ij}$ is the number of true positives and total positives, respectively, for class $i$, classifier $j$, $\alpha$ is an intercept (giving the precision for classifier 1), $x_j=0$ for classifier 1 and $x_j=1$ for classifier 2, and $\beta$ gives the difference in precision between the two classifiers.
In R, this would look like this:
glm(cbind(Ncorrect, Nincorrect) ~ classifier,
data=TableOfClassifierPerformanceByClass, family='binomial')
Comment binomial regression
This would be a good approach if you do not have access to the item-level crosstabs (joint confusion matrix), or if you need to test the precision per se.
Tests on $\mathbf v_1$ $\mathbf v_2$ alone
You could also use a non-parametric test on the vector of precisions alone (thus ignoring the information you have about how it was generated). These may be considerably less powerful than the options identified above (since you effectively only have a sample size of 30, and you don't get to leverage the knowledge that a precision formed over many hundreds of items has small sampling variability).
