Let us say that we have given two random variables with different prob. distributions: A = [0.1, 0, 0.5, ...] B = [0.3, 0.1, 0.03, ...]
What should I do when I want to compute the reformulated cross-entropy between those two? (log base here is 2)
The problem here is when I want to calculate 0.1 * log2(0)....
Should I skip the 0 value and its equivalent prob. from the other set? Or rather change the zero to another scalar?