# Quantifying the difference between micro and macro averaging in cross-validation

I am studying the effect of micro vs macro averaging the results of a cross-validation run. Using true positive rate as a running example, given a cross-validation run of $$K$$ folds, the final $$TPR$$ can be computed by:

• micro-averaging: $$TPR_\mu = \dfrac{\sum_{i=1}^K TP_i}{\sum_{i=1}^K N_i^+}=\dfrac{\sum_{i=1}^K TP_i}{N^+}$$
• macro-averaging: $$TPR_M=\dfrac1K\sum_{i=1}^K \dfrac{TP_i}{N_i^+}$$ where $$N_i^+$$ is the number of positive examples in fold $$i$$ and $$N^+$$ is the total number of positive examples.

Other answers and literature [1,2] argue that $$TPR_\mu$$ is the most unbiased estimator for the $$TPR$$. Here, I am interested in quantifying the difference between $$TPR_\mu$$ and $$TPR_M$$, particularly with respect to the number of positive examples in each fold, to better understand the sources of a potential bias in $$TPR_M$$.

My attempt is the following:

Let $$f_i=\dfrac{N_i^+}{N_i}$$ be the fraction of positives in fold $$i$$, and $$f = \dfrac{N^+}{N}$$ be the fraction of positives in the entire dataset. We have that: \begin{align} N_i^+ &= f_i \cdot N_i = f_i \cdot \dfrac{N}{K} \\ N_i^+ &\sim \mathsf{Binomial} \left( \frac{N}{K}, \ f \right) \\ f_i &\sim \mathcal{N} \left( f, \ \sqrt{\frac{K}{N} \cdot f \cdot (1-f)} \right) \end{align}

Finally, \begin{align} TPR_\mu &= \dfrac{\sum_{i=1}^K TP_i}{\sum_{i=1}^K N_i^+} = \dfrac{1}{N}\sum_{i=1}^K \color{blue}{\dfrac{TP_i}{f}} \\ TPR_M &= \dfrac1K\sum_{i=1}^K \dfrac{TP_i}{N_i^+}=\dfrac1K\sum_{i=1}^K \dfrac{TP_i}{f_i\cdot\frac{N}{K}} = \dfrac{1}{N}\sum_{i=1}^K\color{magenta}{\dfrac{TP_i}{f_i}} \end{align}

So, essentially, the difference between micro and macro averaging is solely due to the fraction of positives in each split. If each split contains the same fraction of positives (i.e. $$f_i = f, \ \forall i$$), then $$TPR_\mu = TPR_M$$.

My questions:

1. Can I quantify some sort of difference between $$TPR_\mu$$ and $$TPR_M$$ given the variance of $$f_i$$? For example, if the variance is high, I expect $$TPR_M$$ to greatly differ from $$TPR_\mu$$, whereas if all $$f_i$$ are close to the mean, there wouldn't be much difference.
2. Are there any interesting insights I can extract from this calculations?