# Bayesian decision boundary for bivariate uniform distributions

I would like an analytical derivation of the Bayesian decision boundary between 2 bivariate uniform distributions. Let me explain with an example. Suppose the two distributions are $$U_1 = 0.25 \text{ if } 0 \le x_1, x_2 \le 2 \text{ else }0$$ and $$U_2 = 0.5 \text{ if } 1 \le x_1 \le 2, 1 \le x2 \le 3 \text{ else }0$$My intuition is the following: the distributions overlap on the square $$1 \le x_1,x_2 \le 2$$ and $$U_2$$ is more probable there so it takes it. That is the boundary is $$\text{sample}(x_1,x_2) \in U_2 \text{ if } x1,x2 > 1$$ So far I have silently assumed that the priors are equal. If that's not the cases then not many things change. The only difference is that if the prior of $$U_1$$ is greater than $$2/3$$ it takes the overlapping square. In other words, the boundary becomes $$\text{sample}(x_1,x_2) \in U_2 \text{ if } x1,x2 > 2$$ How do I prove this though?

PS: My question is kinda the 2d version of this one.

• Sure. I don't think there's something missing anyway. But I have read that on stackexchange sites is a good practice to add similar questions and clarifying the differences in order to not have them wrongfully flagged as duplicate. Perhaps I should add the similar one in the end?
– cgss
Nov 23, 2020 at 12:23
• Oh yes. I would try to fix it.
– cgss
Nov 23, 2020 at 12:32
• @Xi'an How does it look now?
– cgss
Nov 23, 2020 at 12:38
• @Xi'an Given a sample($x_1, x_2$) the Bayesian classifier labels it as category 1 or category 2 depending on which side of the decision boundary it is. I want to formally solve the problem of finding the equation of said boundary.
– cgss
Nov 23, 2020 at 14:53

Since there is no mention of costs associated with decisions, I assume that the Bayesian decision rule is minimizing the average probability of error.

When Hypothesis $$H_0$$ is true, the observation $$(x_1, x_2)$$ is uniformly distributed on a square of side $$2$$ (sides parallel to the axes) and opposite vertices $$(0,0)$$ and $$(2,2)$$.

When Hypothesis $$H_1$$ is true, the observation $$(x_1, x_2)$$ is uniformly distributed on a rectangle of area $$2$$ (sides parallel to the axes) and opposite vertices $$(1,1)$$ and $$(2,3)$$.

Thus, the likeliihood ratio $$\displaystyle \Lambda(x_1,x_2) = \frac{f_1(x_1,x_2)}{f_0(x_1,x_2)}$$ has value $$\displaystyle\frac{0}{0.25} = 0$$ when $$(x_1,x_2)$$ lies in an L-shaped region of area $$3$$, value $$\displaystyle\frac{0.5}{0.25} = 2$$ when $$(x_1,x_2)$$ lies in the square of area $$1$$ with opposite vertices $$(1,1)$$ and $$(2,2)$$, and value $$\displaystyle\frac{0.5}{0} = \infty$$ $$(x_1,x_2)$$ lies in the square of area $$1$$ with opposite vertices $$(1,2)$$ and $$(2,2)$$. Now, the Bayesian decision rule compares $$\Lambda(x_1,x_2)$$ to the threshold $$\displaystyle \frac{\pi_0}{\pi_1}$$ where $$\pi_0$$ and $$\pi_1 = 1-\pi_0$$ are the prior probabilities of $$H_0$$ and $$H_1$$, and so it is easy to figure out what the Bayesian decision rule is, and what the corresponding decision boundary is.

Let $$\Gamma_0$$ and $$\Gamma_1$$ denote the disjoint regions such that when $$(x_1,x_2) \in \Gamma_i$$, the decision is that $$\Gamma_i$$ is the true hypothesis.

• If $$\displaystyle\pi_0 < \frac 23$$ so that $$\displaystyle\frac{\pi_0}{\pi_1} < 2$$, $$\Lambda(x_1,x_2) > \displaystyle\frac{\pi_0}{\pi_1}$$ whenever $$(x_1,x_2)$$ lies in the rectangular region of area $$2$$ (sides parallel to the axes) and opposite vertices $$(1,1)$$ and $$(2,3)$$. Thus, $$\Gamma_1$$ is this rectangular region, and we have \begin{align}\Gamma_1 &= \{(x_1,x_2)\colon x_1 \geq 1, x_2 \geq 1\}\\ \Gamma_0 &= \{(x_1,x_2)\colon x_1 < 1~ \textbf{or} ~ x_2 < 1\}\end{align}

• If $$\displaystyle\pi_0 > \frac 23$$ so that $$\displaystyle\frac{\pi_0}{\pi_1} > 2$$, $$\Lambda(x_1,x_2) > \displaystyle\frac{\pi_0}{\pi_1}$$ whenever $$(x_1,x_2)$$ lies in the square region of area $$1$$ (sides parallel to the axes) and opposite vertices $$(1,2)$$ and $$(2,3)$$. Thus, $$\Gamma_1$$ is this square region, and we have \begin{align}\Gamma_1 &= \{(x_1,x_2)\colon x_1 \geq 1, x_2 \geq 2\}\\ \Gamma_0 &= \{(x_1,x_2)\colon x_1 < 2~, x_2 <2\}\end{align}

• If $$\pi_0 = \frac 23$$ exactly so that $$\Lambda(x_1,x_2)$$ has value $$2$$ exactly equal to the threshold for all $$(x_1,x_2)$$ in the square region of area $$1$$ with opposite vertices $$(1,1)$$ and $$(2,2)$$. In this instance, the average error probability is the same regardless of which points in this region we assign to $$\Gamma_0$$ and which we assign to $$\Gamma_1$$ !! So, we arbitrarily choose to assign them all to $$\Gamma_0$$ resulting in \begin{align}\Gamma_1 &= \{(x_1,x_2)\colon x_1 \geq 1, x_2 \geq 2\}\\ \Gamma_0 &= \{(x_1,x_2)\colon x_1 < 2~, x_2 <2\}\end{align}

If the OP desires, he can massage the above information into a single humongous formula that upon plugging in the values of $$\pi_0, \pi_1$$ gives the exact decision boundaries directly and thereby amaze his friends and TA.

• I was trying to use the rule $U_1\cdot \text{ prior of } U_1 = U_2\cdot \text{ prior of } U_2$. I understand you are using the same rule before cross multiplying, correct? You are using the boundary as known and figuring out the decision. I agree. The problem is I am trying to have the boundary as unknown and the solve the equation to find what I see in my drawings. It's like when we find out the intersection point in the 1d case. Does it makes sense what I am asking?
– cgss
Nov 23, 2020 at 17:48