# manual implementation of Gaussian naive bayesian returns posterior larger than 1

I try to implement Gaussian naive bayesian manually in R. I test my model on iris data set. I would like to build a predictive model. That is, I would like to consider, for each point, the correct class.

## My problem:

I see that the posteriors for some points are larger than 1, which is wrong.

## What I have tried:

• I have divided the data into training and test datasets.

• Then I assume equally prior for each class (0.3) (as the prior must sum to one).

• I applied the Bayesian rule for the training data set. For this I do the following:

• to compute the likelihood for each class, I computed the density (for each point in each class). den[[1]] is the density of the Sepal.Length. den[2] is the density of Sepal.Width. den[[3]] and den[[4]] are the density for the resining variables.

• Then to compute the posterior for each point, I computed prior*likelihood, for each point.

However, I found that some posteriors values are larger than one. I believe I did something wrong, but do not know what is my mistake?

## Here is my code

data(iris)
dim(iris)[[1]]
##split the data based on its class
NewData <- split(iris, iris$$Species) ## divide the data based on their class NewDatSetosa <- NewData$$setosa[,1:4]
NewDatVersicolor <- NewData$$versicolor[,1:4] NewDatVirginica <- NewData$$virginica[,1:4]

##divide the data into train and test datasets
set.seed(1234)
datadivision <- sample(2, nrow(iris), replace=TRUE, prob=c(0.67, 0.33))
##Compute the posterior for each class

posterior <- function(data,prior,newDat){
post <- list()
##compute the density for each point in the training data set
den <- lapply(1:4, function(i) dnorm(data[[i]], mean(newDat[[i]]), sd(newDat[[i]])))
## compute the posterior for each point. den[[1]] is the density of the
##Sepal.Length. den[2] is the density of Sepal.Width. den[3] and den[4] ##are the density for the resining variables.

post <- prior*den[[1]]*den[[2]]*den[[3]]*den[[4]]
return(post)
}
post.Setosa <- posterior(iris.training, prior=0.3, NewDatSetosa)
post.Versicolor <- posterior(iris.training, prior=0.3, NewDatVersicolor)
post.Virginica <- posterior(iris.training, prior=0.3, NewDatVirginica)
• Not sure I understand what you are doing, but why would a normal distribution (not restricted to [0, 1]) for the likelihood make sense, if you expect an posterior to have something to do with a probability of class membership? – Björn Apr 30 '19 at 11:35
• @Björn As I understand the normal distribution must be estimated for each class. I tried to do a classification problem. So, I estimate the normal distribution for each class and then apply the Bayesian rule. – Mary Apr 30 '19 at 11:37

Your priors should be $$1/3$$, not 0.3. But, this is the minor issue. It's true that your posterior, i.e. $$p(c|x)$$, where $$c$$ represents the class can't be larger than $$1$$. However, it's not equal to prior * likelihood; rather it is proportional to it: $$p(c|x)=\frac{p(x|c)p(c)}{p(x)}\propto p(x|c)p(c)$$ So, what you've calculated is not a valid PDF or PMF. It should be normalized by $$p(x)=\sum_{c_i} p(x|c_i)p(c_i)$$. But, you can do the classification w/o the normalization step because $$p(x)$$ doesn't depend on specific $$c_i$$.