As a part of going through the newly release TidyModels with R by Max Kuhn and Julia Silge I dug into some of the references on Chapter 9. More specifically A Unified View of Multi-Label Performance Measures and while making certain I understood the notation, I quickly hit a wall on several of the measurements.
To keep my question focused, I will only ask about one of the measurements mentioned in the article here, coverage.
The article defines the coverage measure of a multi-classification problem is defined as
$$
coverage(F) = \frac{1}{m}\sum_{i=1}^{m}[\![\underset{j\in Y_{i.}^{+}}{\mathrm{max}}\ \mathrm{rank}_F(x_i,j) - 1]\!]
$$
where
- $x_i\in \mathbb{R}^{d\times 1}$ is a real valued vector
- $Y_{i.}^{+} = \{j|y_{ij} = 1\}$ denotes the index set of relevant observations
- $y_{ij}\in\{0, 1\}^{l\times 1}$ is a label vector
- $F\ :\ \mathbb{R}^{d}\rightarrow \mathbb{R}^{l} = \{f_1, \ldots , f_l\}$ is the multi-label predictor function mapping a probability value for $x_i$ for label $y_{ij}$
- $m$ is the number of observations
- $l$ is the number of unique labels in $Y$ where $Y_i = \{y_{i1},\ldots, y_{il}\}$.
and gives a confusing description of the measure as
The number of more labels on aver-age should include to cover all relevant labels
My question becomes how to interpret the above equation, and given a defunct example where $m=1$ how to implement the formula based on their definition. Hopefuly the latter is glaringly obvious given the first.
Defunc example: $m=1$
From the modeldata
package I'll use the first wrongly specified label in the hpc_cv
dataset
library(modeldata)
library(dplyr)
data('hpc_cv', package = 'modeldata')
filter(hpc_cv, obs != pred) %>% select(obs, pred, VF, F, M, L) %>% head(1)
obs pred VF F M L
1 VF F 0.3761772 0.5456339 0.07679576 0.001393113
Repeating the question: How should the above formulation of coverage be interpreted and given the example data above what would the result become.