Two confusion matrices In a machine learning context, I am working on a binary classification problem. There is a source of truth $T$ for labels, and a labeling process $A$ which is not perfect and makes errors compared to $T$, according to a confusion matrix $C_{TA}$. There is then a second labeling process $B$, and a second confusion matrix $C_{AB}$ between $A$ and $B$.
Is it possible to calculate $C_{TB}$ from $C_{TA}$ and $C_{AB}$? If so, what would that calculation be?
 A: It is not possible to calculate $C_{TB}$ from $C_{TA}$ and $C_{AB}$ alone.
As a counter-example, here are two trivial datasets which have the same confusion matrices $C_{TA}$ and $C_{AB}$, but have different $C_{TB}$.
dat1 <- data.frame(matrix(
  c(rep(c("T","A","B"), times = 1),
    rep(c("!T","A","B"), times = 1),
    rep(c("T","!A","B"), times = 1),
    rep(c("!T","!A","B"), times = 1),
    rep(c("T","A","!B"), times = 1),
    rep(c("!T","A","!B"), times = 1),
    rep(c("T","!A","!B"), times = 1),
    rep(c("!T","!A","!B"), times = 1)),
  byrow = TRUE, ncol = 3))
table(dat1$X1, dat1$X2)
#>     
#>      !A A
#>   !T  2 2
#>   T   2 2
table(dat1$X2, dat1$X3)
#>     
#>      !B B
#>   !A  2 2
#>   A   2 2
table(dat1$X1, dat1$X3)
#>     
#>      !B B
#>   !T  2 2
#>   T   2 2

dat2 <- data.frame(matrix(
  c(rep(c("T","A","B"), times = 2),
    rep(c("!T","A","B"), times = 0),
    rep(c("T","!A","B"), times = 1),
    rep(c("!T","!A","B"), times = 1),
    rep(c("T","A","!B"), times = 0),
    rep(c("!T","A","!B"), times = 2),
    rep(c("T","!A","!B"), times = 1),
    rep(c("!T","!A","!B"), times = 1)),
  byrow = TRUE, ncol = 3))
table(dat2$X1, dat2$X2)
#>     
#>      !A A
#>   !T  2 2
#>   T   2 2
table(dat2$X2, dat2$X3)
#>     
#>      !B B
#>   !A  2 2
#>   A   2 2
table(dat2$X1, dat2$X3)
#>     
#>      !B B
#>   !T  3 1
#>   T   1 3

Created on 2022-05-28 by the reprex package (v2.0.1)
If we are able to assume that $P(B|A,T) = P(B|A)$, that is, the probability of $B$ being positive is independent of the true value of $T$ given the value of the first classifier $A$, then we can find the joint distribution of $T, A, B$ from $C_{TA},C_{AB}$, and from that find $C_{TB}$. This assumption is not testable from the two confusion matrices, and would need to be evaluated via domain knowledge.
Suppose the two matrices are
$$
C_{TA} = \begin{bmatrix} 
& !A & A \\
!T & a & b \\
T & c & d
\end{bmatrix}\\
C_{AB}= \begin{bmatrix} 
& !B & B \\
!A & e & f \\
A & g & h
\end{bmatrix}
$$
Where $a + c = e + f$ and $b + d = g + h$. If $P(B|A,T) = P(B|A)$, then the number of observations which are $TAB$ will be $\frac{h}{g+h}(d)$. That is, the 2nd classifier takes the $d$ observations that were $TA$ and classifies them as $B$ with proportion $h/(g+h)$. Similarly the number of observations which are $T!AB$ will be $\frac{f}{e+f}(c)$. Combining gives the number of observations which are $TB$, $\frac{h}{g+h}(d) + \frac{f}{e+f}(c)$. Carrying on gives the desired matrix.
$$
C_{TB}= \begin{bmatrix} 
& !B & B \\
!T & \frac{e}{e+f}a + \frac{g}{g+h}b & \frac{f}{e+f}a + \frac{h}{g+h}b  \\
T & \frac{e}{e+f}c + \frac{g}{g+h}d & \frac{f}{e+f}c + \frac{h}{g+h}d
\end{bmatrix}
$$
This confusion matrix may have non-integer entries, but it's elements are positive and add up to $a+b+c+d$.
Note again that this assumes $P(B|A,T) = P(B|A)$, which is not testable from $C_{TA}$ and $C_{AB}$. This assumption could be plausible if the second classifier is naive to the original classifier's purpose, and is strictly trying to predict the result of the first classifier. For example, if the second labeling process is trying to transmit the classification from A to a third party without making errors.
