# How to find quantiles and likelihoods of mixture distributions?

My PDF:

M was estimated and found to be 5.

I need to work out the quartiles for the PDF above. In addition, I need to use different methods of estimation to estimate the parameters. So far I've successfully used the method of moments. I'm finding it difficult to form the likelihood function.

This is the approach I took to work out the quartiles, which I believe is not correct; quartile 1 below:

And this is what I have so far for the likelihood function:

Any help would be truly appreciated! Thank you.

• Why not typeset here using MathJax? – StubbornAtom Jun 5 '19 at 13:35

It helps to recognize the origin of this distribution: it is a mixture of a uniform distribution $$F_M$$ on the integers $$\{0,1,\ldots,M\}$$ and the Poisson distribution $$G_a$$ of parameter $$a,$$

$$\Pr(X=k\mid M, a, p) = pf_M(k) + (1-p)g_a(k),$$

where, for this particular question,

f_M(k) = \left\{\eqalign{\frac{1}{M+1},&\ k\in\{0,1,\ldots,M\} \\ 0&\ \text{otherwise};}\right.

$$g_a(k) = e^{-a} \frac{a^k}{k!},\ k\in\{0,1,2,\ldots\};$$

and $$0\le p \le 1$$ is the proportion (or weight) of $$F_M$$ and $$1-p$$ is the proportion of $$G_a$$ in the mixture.

This both simplifies and generalizes the notation, enabling us to see past the details to the underlying concepts.

Let's address the questions in reverse, beginning with the easiest: the likelihood. By definition, the likelihood for a dataset of values $$\mathbf{x}=(x_1, x_2, \ldots, x_n)$$ presumed to realize a simple random sample from any distribution is the chance of these values. Since in a simple random sample they are independent, that chance is the product of the individual chances,

$$\mathcal{L}(\mathbf{x}; (M,a,p)) = \prod_{i=1}^n \Pr(X=x_i\mid M,a,p).\tag{1}$$

Each data value $$x_i$$ contributes its term to this product.

If you would like a more explicit expression, the problem arises that $$\Pr(X=x\mid M,a,p)$$ is given with two formulas, conditional on whether $$x\le M$$ or $$x\gt M.$$ One solution is to separate the product $$(1)$$ into a product over the $$x_i$$ with values up to $$M$$ and another product over the other values:

$$\mathcal{L}(\mathbf{x}; (M,a,p)) = \prod_{i:\,x_i\le M}\left(\frac{p}{M+1}+(1-p)e^{-a}\frac{a^{x_i}}{x_i!}\right)\prod_{i:\,x_i\gt M}\left((1-p)e^{-a}\frac{a^{x_i}}{x_i!}\right).$$

Finding quantiles is harder. By definition, the quantile $$x_{(q)}$$ corresponding to a probability $$0\le q \le 1$$ is the smallest value $$x$$ for which the distribution function equals or exceeds $$q.$$ The distribution of the mixture is the weighted value of its component distributions, $$pF_M + (1-p)G_a.$$ Thus

$$(pF_M + (1-p)G_a)(x_q) \ge q\ \text{ and whenever }x\lt x_q,\ (pF_M + (1-p)G_a)(x) \lt q.\tag{2}$$

A useful way to restate this is that $$x_{(q)}$$ is a zero of the function

$$x\to (pF_M + (1-p)G_a)(x) - q.$$

This enables us to use an appropriate root finding or minimization routine to find quantiles. There is, in general, no explicit or simple formula for them in terms of the quantile functions of $$F_M$$ and $$G_a.$$ What we can do is to restate $$(2)$$ in the useful form

$$(pF_M + (1-g)G_a)(x_{(q)}-1) \lt q \le (pF_M + (1-g)G_a)(x_{(q)}).\tag{2a}$$

We can illustrate $$(2a)$$ by plotting the CDF. Here, in one place, are plots of $$pF_M$$ (the uniform distribution, in gold), $$(1-p)G_a$$ (the Poisson distribution, in light blue), and the mixture (in gray). I have chosen $$M=5$$ as in the question and, for this illustration, arbitrarily set $$a=3$$ and $$p=1/3.$$

The intersecting dotted lines indicate the third quartile. It was found by setting $$q=3/4$$, finding the solution to $$(2a),$$ and plotting a horizontal line at height $$3/4$$ and a vertical line at the position $$x_{(3/4)} = 4.$$ The height of the red dot on that vertical line is, of course, the value of the mixture CDF, $$(1/3)F_5(4) + (1-1/3)G_3(4).$$ It exceeds $$3/4,$$ but you can see this occurs at a step where all values to its left are less than $$3/4.$$

Because it may be instructive, here is the R code used to generate the figure. It can be applied (within the limits of floating point computation) to any combination of the parameters $$m,a,p$$ you may choose subject to the necessary mathematical restrictions on their values (all are nonnegative, $$m$$ must be integral, and $$p$$ cannot exceed $$1$$).

#
# Find the quantile of any CDF f.
#
# This is a little tricky for discrete f.  The tangent enables us to specify a
# finite search interval.  Due to floating point error, the solution returned
# by uniroot may be not quite an integer and therefore the integers on
# both sides of it need to be checked.
#
q.generic <- Vectorize(function(q, f, ...) {
obj <- function(x) f(x, ...) - q
sol <- uniroot(function(u) {obj(tan(u))}, c(-pi/2, pi/2), f.lower=-q, f.upper=1-q)
if(sol$$f.root + q <= 0) return(-Inf) # if(sol$$f.root + q >= 1) return(Inf)
x <- tan(sol\$root)
y <- obj(x <- c(floor(x), ceiling(x)))
ifelse(y[1] >= 0, x[1], x[2])
}, "q")
#
# Define the CDFs of the mixture components and the mixture.
#
F.cdf <- function(k, m) stepfun(seq(m+1)-1, seq(0,1,length.out=m+2))(k)
G.cdf <- function(k, a) ppois(k, a)
pMix <- function(x, m, a, p) p * F.cdf(x, m) + (1-p) * G.cdf(x, a)
#
# Specify parameters.
#
m <- 5
p <- 1/3
a <- 3
#
# Plot the CDF of the mixture and its components.
#
x.max <- max(m+1, a + 3*sqrt(a))
plot(c(-1, x.max), c(0,1), type="n", ylab="Probability", xlab="k", bty="n",
main="Mixture CDF, Its Components, and Q3")
curve(p * F.cdf(x, m), type="s", n=501, add=TRUE, col="Tan", lwd=2)
curve((1-p) * G.cdf(x, a), type="s", n=501, add=TRUE, col="SkyBlue", lwd=2)
curve(pMix(x, m, a, p), n=501, type="s", add=TRUE, col="#00000080", lwd=2)
points(seq(x.max+1)-1, pMix(seq(x.max+1)-1, m, a, p), pch=21, bg="Gray")
#
# Find the quartiles of the mixture.
#
quartiles <- q.generic(seq(1/4,1,by=1/4), pMix, m, a, p)
#
# Show the third quartile on the plot.
#
abline(h=c(3/4), lty=3)
abline(v=quartiles[3], lty=3)
points(quartiles[3], pMix(quartiles[3], m, a, p), pch=21, cex=1.2, bg="Red")

• Thank you so much ! I thought about that form of the likelihood but was not sure. Thanks again ! :D – Francesca Camilleri Jun 5 '19 at 17:26