This is for my hw, and if anyone can solve the first part of the question it will be great. Here is the question:

Assume a two-class problem with equal a priori class probabilities and Gaussian class-conditional densities as follows:

$$p(x\mid w_1) = {\cal N}\left(\begin{bmatrix} 0 \\ 0 \end{bmatrix},\begin{bmatrix} a & c \\ c & b \end{bmatrix}\right)\quad\text{and}\quad p(x\mid w_2) = {\cal N}\left(\begin{bmatrix} d \\ e \end{bmatrix},\begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}\right)$$

where $ab-c^2=1$.

  1. Find the equation of the decision boundary between these two classes in terms of the given parameters, after choosing a logarithmic discriminant function.
  2. Determine the constraints on the values of a, b, c, d and e, such that the resulting discriminant function results with a linear decision boundary.
  • 3
    $\begingroup$ I see a study group forming... $\endgroup$
    – cardinal
    Nov 13 '11 at 15:41
  • $\begingroup$ i don't understand what you mean but, if this is the exact duplicate of a question asked before, i can't see the question or answer. Where can i find? $\endgroup$
    – ceylan
    Nov 13 '11 at 17:06
  • $\begingroup$ See the link to "Possible duplicate" at the top of this question? $\endgroup$
    – cardinal
    Nov 13 '11 at 17:10
  • 2
    $\begingroup$ Should this question be re-opened since the one that it duplicates has been removed voluntarily by the author, or should the author be asked to consider removing this one too? $\endgroup$ Nov 14 '11 at 12:41
  • $\begingroup$ @Dilip: Under the circumstances, I would agree it should probably be reopened. $\endgroup$
    – cardinal
    Nov 14 '11 at 20:29

I will use $(X,Y)$ for the observation. Given $w = w_1$, we have that the variances of $X$ and $Y$ are $a$ and $b$ respectively, while the covariance is $c$. Thus, the correlation coefficient $\rho = \frac{c}{\sqrt{ab}}$ and so $1-\rho^2 = 1 - \frac{c^2}{ab} = \frac{1}{ab}$. The two conditional (joint) densities are joint normal densities given by $$\begin{align} f_1(x,y) &= \frac{1}{2\pi \sqrt{ab(1-\rho^2)}}\exp\left[-\frac{1}{2(1-\rho^2)}\left(\frac{x^2}{a} - 2\rho\frac{xy}{\sqrt{ab}} + \frac{y^2}{b}\right)\right]\\ &= \frac{1}{2\pi}\exp\left[-\frac{ab}{2}\left(\frac{x^2}{a} - 2\frac{c}{\sqrt{ab}}\frac{xy}{\sqrt{ab}} + \frac{y^2}{b}\right)\right]\\ &= \frac{1}{2\pi}\exp\left[-\frac{1}{2}\left(bx^2 - 2cxy + ay^2\right)\right],\\ f_2(x,y) &= \frac{1}{2\pi}\exp\left[-\frac{1}{2}\left((x-d)^2+(y-e)^2\right)\right]\\ \end{align}$$ The decision boundary is the set of all $(x,y)$ for which $f_1(x,y) = f_2(x,y)$, and so the decision boundary is the conic section specified by $$bx^2 -2cxy +ay^2 - \left((x-d)^2 + (y-e)^2\right) = 0.$$ Edit: As pointed out in the comment by whuber, this conic section can be a hyperbola as well as an ellipse or parabola (including as a special case a straight line). The discriminant $c^2-(a-1)(b-1) = (a+b)-2$ (since we are given that $ab-c^2=1$) can be positive, negative, or zero depending on the variances $a$ and $b$. I suspect that this will be work out to be either an ellipse or a parabola (but not a hyperbola) depending on the parameters $a,b,c,d,e$, including, as a special case of parabola, a straight line. A specific example of a straight-line decision boundary is when $c = 0$ and $a = b = 1$ so that the only difference between the two conditional distributions is the means: $X$ and $Y$ are conditionally independent unit-variance normal random variables under either hypothesis. In this instance, the decision boundary is the perpendicular bisector of the straight line segment with end-points $(0,0)$ and $(d,e)$.

What puzzles me, though, is a hyperbola as a decision boundary since a hyperbola partitions the plane into three regions (two of which are congruent). The joint density surfaces are a (possibly) flattened bell and a circular bell, and so one of these bells subsumes the other in two non-contiguous regions of the plane: I just can't visualize it. Perhaps someone will create a nice illustration....

  • $\begingroup$ Good beginning! I fixed a subtraction error in the final formula. As a result, you might want to revisit your speculations in the last paragraph. To get started, note that the equation is linear if and only if all the quadratic coefficients are zero, which is equivalent to $a=1$, $b=1$, and $c=0$. (There is another special case where the equation represents a pair of lines.) Whether the locus is elliptical or hyperbolic depends on the sign of the determinant $(a-1)(b-1)-c^2$; both signs are possible. $\endgroup$
    – whuber
    Mar 17 '13 at 16:06
  • $\begingroup$ Re the visualization: Imagine the intersection of a spherical bowl with a bowl that is so long and shallow (but with steep sides) that it looks almost like a folded piece of paper. That is what things look like when $b=2/a$ and $c=1$. The steep sides of the paper rise above the bowl's sides, but the valley along the center of the paper falls below the bowl's sides. Thus, the region where the paper is higher consists of two separate areas delimited by the arcs of the hyperbola. $\endgroup$
    – whuber
    Jan 8 '15 at 3:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.