Suppose, I'm doing a classification with two classes in $\mathbb{R^d}$. The true class-conditional probability distribution of two classes are multivariate Gaussian with mean $\mathbf{\mu_1}$ and $\mathbf{\mu_2}$ but same covariance $ \sigma^2 \mathbf{I}$. What will be the Bayes risk (error rate) of this Bayes classifier? Is there a closed form formula? If not, how does it depend on $d$ i.e does it increase or decrease with $d$, dimension of the space? or What about $\sigma$?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
The Bayes risk can be computed explicitly even in the case where the common covariance matrix $\Sigma$ is not a multiple of the identity matrix. If $\pi_i$ denotes the probability of choosing an item from class $i \in \{1,2\}$, so that $\pi_1 + \pi_2 = 1$, and if $$\delta = \big\{(\mu_1-\mu_2)^T \Sigma^{-1} (\mu_1-\mu_2)\big\}^{1/2}$$ denotes the Mahalanobis distance between the two classes, then the Bayes risk is equal to $$ \pi_1\Phi\big(-\frac{1}{2}\delta + \frac{1}{\delta}\log(\pi_2/\pi_1)\big) + \pi_2\Phi\big(-\frac{1}{2}\delta - \frac{1}{\delta}\log(\pi_2/\pi_1)\big)\,,$$ where $\Phi(\cdot)$ is the cumulative distribution function for a standard normally distributed variable. See for example Brian Ripley, Pattern Recognition and Neural Networks, various editions, Oxford University Press, for the (short) proof and some commentary.
