Bayes classifier Given the prior probability of 2 distributions, $N(x,y)$ and $N(a,b)$, where $N(\mu,\sigma^2)$:
How do you make a decision rule to minimize the probability of error, if the prior probabilities are equal? Can you give an example?
What if the prior probabilities are different, such as
$P(\text{Distribution 1}) = 0.70$
$P(\text{Distribution 2}) = 0.30$?
 A: Let's say we have to classes $C$, i.e. $c_1$ and $c_2$ and further that the points of the class $c_1$ are drawn from N(a,b) and the points of $c_2$ from N(c,d) respectively (note the changes in notation). The point variable shall be X, a value of this variable x. 
Here is the Bayes-Theorem:
$p(C|X)=\frac{p(X|C)*p(C)}{p(X)}$
The bayes decision rule for binary classification states: Given x, one decides...


*

*in favor of $c_1$ If $p(c_1|x) > p(c_2|x)$ 

*in favor of $c_2$ If $p(c_2|x) > p(c_1|x)$ 


The cases that both posterior probabilities is equal has to be treated separately. Normally one favors one class more than the other and hence decides in favor of that class (if a decision is forced). Let's say, that if the posteriors are equal, we decide in favor of class $c_2$
The above decision rule can be reformulated to:
If $\frac{p(c_1|x)}{p(c_2|x)} > 0$, decide in favor of $c_1$, else $c_2$
Applying Bayes Theorem ...
$\frac{p(c_1|x)}{p(c_2|x)}$ = $\frac{p(x|c_1)*p(c_1)}{p(x|c_2)*p(c_2)}$
If the priors are equal, the decision rule resolves to $\frac{p(x|c_1)}{p(x|c_2)}$. 
If $p(c_1)=0.7$ and $p(c_2)=0.3$, the decision rule resolves to $\frac{0.7}{0.3}*\frac{p(x|c_1)}{p(x|c_2)}\approx 2.33*\frac{p(x|c_1)}{p(x|c_2)}$
The conditional densities p(X|C) are defined by the corresponding normal distributions. The density of a normal distribution for (not at) a point x can be calculated as $pdf(x|N(a,b))=\frac{1}{\sqrt{2\pi*b}} * e^{-\frac{(x-a)^2}{2b}}$. 
Inserting this into our formula (for case of equal priors) one gets 
$\frac{p(x|c_1)}{p(x|c_2)}=\frac{pdf(x|N(a,b))}{pdf(x|N(c,d))}$. Starting from here you can "simplify" the formula even further.
Edit:
Here is some R - code to visualize the priors * conditional probabilities (ignoring the normalization factor $p(X)$):
require(lattice)
#
set.seed(42)

# parameters for distribution of class 1
a <- 0
b <- 1
# parameters for distribution of class 2
c <- 3
d <- 1

x <- c(sort(rnorm(1000,mean=a,sd=b)),sort(c(rnorm(1000,mean=c,sd=d))))
y <- c(dnorm(x[1:1000],mean=a,sd=b),dnorm(x[1001:2000],mean=c,sd=d))
labels <- factor(rep(c("class 1","class 2"),each=1000))

dat <- data.frame("x"=x,"density"=y,"groups"=labels)
xyplot(density~x,data=dat,groups=labels,type="b",auto.key=T)

which results in this plot, where you can see the decision "rule" (line). Starting from here, I suggest to play around a little bit with the priors to get a good feeling "what's going on". 

