# Finding the point of maximum probability in a mixture of gaussians

I have a model that estimates probability of an object to be located in a 2d space. Using a mixture of gaussian with a set of criteria that I chose I got interesting results, and now I am faced to a further problem: finding the point of maximum probability.

I wonder, is finding the maximum point of probability in a mixture of gaussians an analytically tractable problem or search methods need to be used?

If you have a mixture of $n$ one-dimensional Gaussians, you can have anything between one and $n$ local maxima:

weights <- c(0.2,0.3,0.5)
weights <- weights/sum(weights)
sds <- c(1,2,3)

means <- c(-1,0,1)
xx <- seq(min(means-3*sds),max(means+3*sds),by=0.01)
plot(xx,
rowSums(mapply(dnorm,mean=means,sd=sds,MoreArgs=list(x=xx))),
type="l",xlab="",ylab="")


means <- c(-5,0,5)
xx <- seq(min(means-3*sds),max(means+3*sds),by=0.01)
plot(xx,
rowSums(mapply(dnorm,mean=means,sd=sds,MoreArgs=list(x=xx))),
type="l",xlab="",ylab="")


And of course, (some of) these maxima may have the same height, so you may not only have multiple local maxima, but multiple global ones.

As Martin O'Leary very helpfully points out, linking to this extremely helpful page, this does not hold in higher dimensions. Instead, you can easily have more than $n$ maxima in higher dimensions.

Consequently, you won't get around searching for your maxima. Use multiple starting values and a simple Newton-type maximizer over the mixture density, evaluate the density at each local maximum you find, and output the local maximum with the highest density.

In one dimension, if in the course of your search you have found $n$ different local maxima, then you know that the one with the highest density is one of your global maxima. If not, you either have fewer local maxima than $n$ (as in the examples above), or you may have missed some (and the global maximum among them). As per above, this check will not work in higher dimensions.

• It's unlikely to have a huge practical effect, but I should point out that in two or more dimensions, it's possible for a mixture of $n$ Gaussians to have more than $n$ local maxima. This page has more details. – Martin O'Leary May 6 '15 at 13:22
• @MartinO'Leary: thanks, very good point! Let me just edit the answer to include this... – Stephan Kolassa May 6 '15 at 13:42