I have problems with this exam problem (see image below) for machine learning.


For my answer i assumed that $\hat{y}^{}_{0}$ and $\hat{y}^{}_{1}$ are the predicted classes. That means that $\hat{y}^{}_{0}$ predicted class 0 which means $a$ and $\hat{y}^{}_{1}$ predicted class 1 which means $b$.

Furthermore, it is given that both $\hat{y}^{}_{0}$ and $\hat{y}^{}_{1}$ minimize the expected loss.

The exprected loss is:

$\sum^{K}_{k=1} e(\hat{y}, c_k) p_k$

Minimzing exprected loss means that $\hat{y}$ predicts the class $c_k$ with the highest probability $p_k$. Because both $\hat{y}^{}_{0}$ and $\hat{y}^{}_{1}$ minimize the expected loss means that the probability of $p_1$ and $p_2$ have to be the same (and bigger than $p_3$).

Resulting in $p_1 = p_2$ and $p_3 = 1 - (p_1 + p_2)$, $p_3 < p_2$.

Are my assumptions correct?

I was a bit confused about the naming. Why is it not called $\hat{y}^{}_{a}, \hat{y}^{}_{b}$ instead of $\hat{y}^{}_{0}, \hat{y}^{}_{1}$ and $(p_0, p_1, p_2)$ instead of $(p_1, p_2, p_3)$?


1 Answer 1


I would have thought that

  • $\hat{y}^{}_{0}$ is the estimator which minimises the $0/1$ loss
    • as you say, this is class with the highest conditional probability $p_k$
  • $\hat{y}^{}_{1}$ is the estimator which minimises the $L_1$ loss
    • this should the median of the conditional distribution

As an example of where they might produce different results, consider $(p^{}_1, p^{}_2, p^{}_3) = (0.35, 0.25, 0.4)$, which would give $\hat{y}^{}_{0} = c$ but $\hat{y}^{}_{1} = b$.

As for notation, the subscripts in $\hat{y}^{}_{0}$ and $\hat{y}^{}_{1}$ are arbitrary though vaguely suggestive of the two loss functions, and they are nothing to do with $a,b,c$. Perhaps in another context you could similarly use $\hat{y}^{}_2$ for the estimator for the $L_2$ expected loss minimising estimator, which would be the mean of the conditional distribution

On the other hand, the subscripts in $p^{}_1, p^{}_2, p^{}_3$ (perhaps suggesting the ordering) could easily have been written as $p^{}_a, p^{}_b, p^{}_c$ here

  • $\begingroup$ Oh wow. Thanks for pointing that out. That makes a lot more sense. $\endgroup$
    – Jens
    Commented Aug 12, 2018 at 11:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.