2
$\begingroup$

$$E[L] = \sum_k \sum_j \int_{R_j} L_{k,j} p(x, C_k)$$

L is a loss function that returns a real value given a pair (i,j), with i as the index of true class, and j as the index of the predicted class of an input. $p(x, C_k)$ is a joint pdf of a random variables X, which is the input, and C, which is a class. $R_j$ is a decision region j of the input space. My question is how to interpret the right side of the equation as an expectation? L is not a function of x, and so $p(x, C_k)$ is not a corresponding distribution. I expect the relevant pdf should be with respect to the random vector $(C, D)$ with C is the predicted class, and D is the true class of an input.

$\endgroup$
1
  • $\begingroup$ Any integral is an expectation: this is the principle for importance sampling. $\endgroup$
    – Xi'an
    Jan 26 at 6:30
2
$\begingroup$

$L$ is indeed a function of $X$, i.e. data. Without data, you can't calculate your loss. $L$ doesn't depend on $X$ if you have the predicted class, but the predicted class itself is a function of input $X$. So, $E[L]$ should include both the data and the class. Also, the expression is correct when $C_k$ is the true label, not predicted.

The inside integral without the loss term is basically the probability of obtaining a sample from $R_j$, i.e. predicting the class as $j$ when the true label is $C_k$: $$\int_{R_j}p(x,C_k)dx=p(C_k)\int_{R_j} p(x|C_k)dx=p(C_k)p(D_j|C_k)=p(D_j,C_k)$$ where $D_j$ denotes the predicted class.

The expectation then becomes $$E[L]=\sum_k\sum_j L_{kj} p(D_j,C_k)$$ which is intuitive since it's over the joint of predicted and true classes.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.