How to compute a 'pair confusion matrix'? I don't really understand how the pair confusion matrix (used for example in comparing of clusterings) is calculated...
pair_confusion_matrix([0, 0, 1, 1], [0, 0, 1, 1]) 
>>> array([[8, 0], 
[0, 4]]) 

Going by the definition here link 1 and here link 2
the upper left entry of the returned 2 by 2 matrix is the number of true negatives, and the lower right entry is the number of true positives.
Where:
TP true positives = number of pairs of samples that are clustered together, and
TN true negatives = number of pairs with both clusterings having the samples not clustered together
But if I were to count here, there are only 2 pairs of samples that are clustered together and only 4 pairs of samples not clustered together.

*

*TP: 0 and 0 + 1 and 1

*TN: 4 combinations of 0 and 1 (i.e. 1st 0 with 1st 1, 1st 0 with 2nd 1, 2nd 0 with 1st 1, 2nd 0 with 2nd 1)


edit 25.10.2021
Going again by the example of two partitions / classifications U and V, where U = [0, 0, 1, 1] and V = [1, 1, 0, 0] for N = 4 objects which I denote as n1, n2, n3 and n4 below.
Based on ttnphns's answer:
If a pair is found in one group in U and is found

*

*in one group in V => goes to a

*not in one group in V => goes to b

If a pair is found not in one group in U and is found

*

*in one group in V => goes to c

*not in one group in V => goes to d

then we have pairs ...
(n1, n2) together in U, and also together in V
(n3, n4) together in U, and also together in V
=> a = 2
(n1,n3) not together in U, and also not together in V
(n1,n4) not together in U, and also not together in V
(n2,n3) not together in U, and also not together in V
(n2,n4) not together in U, and also not together in V
=> d = 4
=> b and c both = 0
so the matrix would look like
[[2, 0],
[0, 4]]
with sum of all entries = 6 = 4C2 (4 choose 2) = N(N-1)/2
But the problem is, that for this exact example the sklearn documentation for their pair_confusion_matrix returns a pair confusion matrix of
[[8, 0],
[0, 4]]
which doesn't makes sense for me at all. Even the sum of all entries is not equal to N(N-1)/2 anymore. The sum 12 which is 24/2 does't even correspond to any nCr value possible since there's no N(N-1) = 24.
 A: Per sklearn documentation I understand it this way.
Let say there are 2 label clusterings: true and pred
label_true = [0, 0, 1, 1, 2, 1]
label_pred = [0, 0, 1, 2, 1, 2]

If we run it with pair_confusion_matrix(label_true, label_pred) from sklearn we will get
array([[20,  2],
       [ 4,  4]])

So how do we get this matrix?
There are 6 objects to be clustered. We can show it like this
                    1obj.  2obj.  3obj.  4obj.  5obj.  6obj.
true clustering => [0,     0,     1,     1,     2,     1]
prediction      => [0,     0,     1,     2,     1,     2]

The left upper corner in the confusion matrix is C_00:
From sklearn doc. it is

number of pairs with both clusterings having the samples NOT clustered
together

We have to count every possible pair not clustered together in 'true clustering' and in 'prediction'. If this pair is not clustered together in both, we will add +2 to C_00. Why 2 rather than 1? I think it is because the permutation matters here.
For example: 1obj. and 3obj. are not in the same cluster in the 'true clustering' and 'prediction'. So +2. But when we check 3obj. and 5obj - not suitable because they are in the same cluster in prediction.
All pairs for C_00: 1obj.3obj.;1obj.4obj.;1obj.5obj.;1obj.6obj.;2obj.3obj.;2obj.4obj.;2obj.5obj.;2obj.6obj.;4obj.5obj.;5obj.6obj.
The right bottom corner in the confusion matrix is C_11:

number of pairs with both clusterings having the samples clustered
together

Both true clustering and prediction have to have elements in the same clusters. Clusters between clusterings may be different. But objects must be in the same clusters within one clustering.
For example: 4obj.6obj. is a candidate to add +2 in the matrix. These are in cluster 1 within true clustering and in cluster 2 within prediction.
All pairs for C_11: 1obj.2obj.;4obj.6obj.;
The left bottom corner in the confusion matrix is C_10:

number of pairs with the true label clustering having the samples
clustered together but the other clustering not having the samples
clustered together

We will count pairs with objects from one cluster in true clustering but these objects have to be from different clusters in prediction
For example: 3obj.4obj. is from within one cluster in true clustering but not in prediction
All pairs for C_10: 3obj.4obj.; 3obj.6obj.
The left bottom corner in the confusion matrix is C_01:

number of pairs with the true label clustering not having the samples
clustered together but the other clustering having the samples
clustered together

We will count pairs with objects from different clusters in true clustering but these objects have to be from one cluster in prediction
For example: 3obj.5obj. is from within one cluster in prediction but not in true clustering
All pairs for C_01: 3obj.5obj.;
The matrix from the first post can be calculated with the same algorithm.
I hope it helps
