How to compute F1-score when there are NA in the vectors? I am using the f1-score (f1 = 2 * (precision * recall) / (precision + recall) to compute the similarity between two vectors (let's call them actual and pred). There are nevertheless some missing values in these vectors:
actual = [-1,1,'NA',1,1,1,1,0,1,-1]
pred   = ['NA',1,1,1,1,-1,0,1,'NA',-1]

As simple as it can get, I am using the f1 formula:
from sklearn.metrics import f1_score:
f1= f1_score(actual,pred,average='micro')

What is the correct way to deal with these NA values? I thought about using listwise correction to remove the row comparisons when NA appears but I am certainly losing some information.
 A: Assuming that actual and pred can both take values in $\{-1,0,1\}$, one way impute the missing values is as follows.
First, remove all the observations with at least one missing value; let's call this the filtered dataset. Estimate the joint frequency distribution of actual and pred variables, i.e. get the two-way table of joint relative frequencies using the filtered dataset.
Suppose the estimated two-way table is
$$\begin{array}{c|ccc|c} 
& & \text{pred}\\
\text{actual}  & -1 & 0  & 1 & Total\\ \hline
-1 & f_{11} & f_{12} & f_{13} &f_{1\bullet} \\ 
0 & f_{21} & f_{22} & f_{23} &f_{2\bullet}  \\
1 & f_{31} & f_{32} & f_{33} &f_{3\bullet}  \\\hline
 & f_{\bullet1} & f_{\bullet2} & f_{\bullet3} &f_{\bullet\bullet}=1  \\
\end{array}$$
Now let's assume that we have an observation with missing pred and actual taking value $-1$. Compute the normalised frequencies
$$\tilde{f}_{11} = f_{12}/f_{1\bullet},\,\,\tilde{f}_{12} = f_{12}/f_{2\bullet},\,\,\tilde{f}_{13}=f_{13}/f_{3\bullet};$$
note that $\tilde f_{12}+\tilde f_{12}+\tilde f_{13}=1.$
Impute a value for pred by drawing randomly from $\{-1,0,1\}$ with cell probabilities $(\tilde f_{12},\tilde f_{12},\tilde f_{13}).$
Clearly, there must be some independence between actual and pred for this approach to be useful. If there is no dependence between these variables you can sample directly from the relative marginal frequency distribution.
