Consider a binary classification task in 2D. Assume that the likelihood of both classes is Gaussian with uniform class prior, how should be the means $(\mu_1,\mu_2)$ and covariances $(\Sigma_1, \Sigma_2)$ of the discriminants $g_{y_1}(x)$ and $g_{y_1}(x)$ such that the decision surface is a spherical? For example, with equal isotropic covariance matrices, by equating both discriminants functions, we have a linear decision surface in the form $Ax + B =0$. My guess is to just take two differents means and covariance matrices in order to have a quadratic decision surface in the form $x'Ax + B'x +c = 0$. Can someone confirm/explain if wrong? Thanks
$\begingroup$
$\endgroup$
1
-
$\begingroup$ Read this answer for a discussion of the simpler case of two variables that are independent Gaussian under one hypothesis, and correlated jointly Gaussian under the other hypothesis. The decision boundary can be a parabola or a hyperbola. Your case is more complicated since the variables are correlated jointly Gaussian under both hypotheses. $\endgroup$– Dilip SarwateCommented Jan 21, 2018 at 18:17
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
In the 2D case, you mean a circular decision boundary, not spherical, correct? This is the simplest way I can think of to accomplish that. Here's a plot I made showing the maximum of two bivariate gaussians:
Both have mean (0, 0). The red one has covariance matrix:
$$\begin{pmatrix}1 & 0\\\ 0 & 1\end{pmatrix}$$
And the blue one has covariance matrix:
$$\begin{pmatrix}0.5 & 0\\\ 0 & 0.5\end{pmatrix}$$
Using these gaussians as your classes would give you a circular decision boundary. If the means were different, or the covariance matrices were full (rather than diagonal), you would end up with a more elliptical decision boundary.
