0
$\begingroup$

Suppose, I'm doing a classification with two classes in $\mathbb{R^d}$. The true class-conditional probability distribution of two classes are multivariate Gaussian with mean $\mathbf{\mu_1}$ and $\mathbf{\mu_2}$ but same covariance $ \sigma^2 \mathbf{I}$. What will be the Bayes risk (error rate) of this Bayes classifier? Is there a closed form formula? If not, how does it depend on $d$ i.e does it increase or decrease with $d$, dimension of the space? or What about $\sigma$?

$\endgroup$
1
$\begingroup$

The Bayes risk can be computed explicitly even in the case where the common covariance matrix $\Sigma$ is not a multiple of the identity matrix. If $\pi_i$ denotes the probability of choosing an item from class $i \in \{1,2\}$, so that $\pi_1 + \pi_2 = 1$, and if $$\delta = \big\{(\mu_1-\mu_2)^T \Sigma^{-1} (\mu_1-\mu_2)\big\}^{1/2}$$ denotes the Mahalanobis distance between the two classes, then the Bayes risk is equal to $$ \pi_1\Phi\big(-\frac{1}{2}\delta + \frac{1}{\delta}\log(\pi_2/\pi_1)\big) + \pi_2\Phi\big(-\frac{1}{2}\delta - \frac{1}{\delta}\log(\pi_2/\pi_1)\big)\,,$$ where $\Phi(\cdot)$ is the cumulative distribution function for a standard normally distributed variable. See for example Brian Ripley, Pattern Recognition and Neural Networks, various editions, Oxford University Press, for the (short) proof and some commentary.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.