Explain equation 1.80 in Pattern Recognition and Machine Learning, Bishop

Question

$$E[L] = \sum_k \sum_j \int_{R_j} L_{k,j} p(x, C_k)$$

L is a loss function that returns a real value given a pair (i,j), with i as the index of true class, and j as the index of the predicted class of an input. $p(x, C_k)$ is a joint pdf of a random variables X, which is the input, and C, which is a class. $R_j$ is a decision region j of the input space. My question is how to interpret the right side of the equation as an expectation? L is not a function of x, and so $p(x, C_k)$ is not a corresponding distribution. I expect the relevant pdf should be with respect to the random vector $(C, D)$ with C is the predicted class, and D is the true class of an input.

Answer 1

$L$ is indeed a function of $X$, i.e. data. Without data, you can't calculate your loss. $L$ doesn't depend on $X$ if you have the predicted class, but the predicted class itself is a function of input $X$. So, $E[L]$ should include both the data and the class. Also, the expression is correct when $C_k$ is the true label, not predicted.

The inside integral without the loss term is basically the probability of obtaining a sample from $R_j$, i.e. predicting the class as $j$ when the true label is $C_k$: $$\int_{R_j}p(x,C_k)dx=p(C_k)\int_{R_j} p(x|C_k)dx=p(C_k)p(D_j|C_k)=p(D_j,C_k)$$ where $D_j$ denotes the predicted class.

The expectation then becomes $$E[L]=\sum_k\sum_j L_{kj} p(D_j,C_k)$$ which is intuitive since it's over the joint of predicted and true classes.