Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Suppose we're given data from three different classes which are normally distributed with the following means and variances:

$C_1: \mu_1=(1,2)^T, \Sigma_1^{-1}=( \begin{array}{ccc}2 & 1 \\1 & 2 \end{array})$
$C_2: \mu_2=(2,-2)^T, \Sigma_2^{-1}=( \begin{array}{ccc}1 & 0 \\0 & 2 \end{array})$
$C_3: \mu_3=(1,-1)^T, \Sigma_3^{-1}=( \begin{array}{ccc}7 & 5 \\5 & 6 \end{array})$

And the loss function of those three classes is $L=\left (\begin{array}{ccc}0 & 1 &3 \\2 & 0 & 2\\ 4&3&0\end{array}\right)$

How should criterion be modeled to make optimal decisions for such a problem? And to which class the point $(0,0)^T$ must be assigned?

I think I can solve the problem without considering the loss function (from prior and posterior probabilities) but I have no idea what can be done about the loss function and consider it in making optimal decisions.

share|improve this question
Is this homework? If so, please add the homework tag so that we can treat it accordingly. – MånsT Jan 8 '13 at 16:39
@MånsT: Done, thank you for reminding me. – Gigili Jan 8 '13 at 17:08
up vote 1 down vote accepted

You need the loss function because it tells you how to penalize each misclassification. What you want to do is compute the expected Bayesian risk: $r(a, \pi) = \int_{\theta} L(\theta, a)\pi(\theta|x)d\theta$, where L() is the loss function and $\pi()$ is the posterior distribution. The loss function is indexed by the action, $a$, this is how you classify the data point, and $\theta$, the actual class of the object. This tells us what our expected penalty is for taking action $a$ when our posterior beliefs are $\pi$.

For any data point, you want to assign the data to the class in which the risk is minimized. There are a few resources online that I've found pretty useful by searching for "optimal Bayesian classification" or "bayesian risk" (e.g. these course notes).

share|improve this answer
I don't understand how to use the formula you suggested. I have calculated $P(x|C_i)$, but what's $L(x,a)$?Or more importantly, what's $a$ here? – Gigili Jan 9 '13 at 6:17
'a' is the action you take. In this case it's making a decision to classify a point (say (0,0)) to one of your three classes. – Pat Jan 9 '13 at 9:39
Right, got it. Thank you @Pat. – Gigili Jan 10 '13 at 19:37

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.