Estimating a cumulative distribution function from a mixture model

I have a problem I fail to find a solution.

Assume a mixture model :

$F(x)=a\times G(x)+(1-a)\times H(x)$

where $F$, $G$ and $H$ are cumulative distributions of different random variables, and $a$ is a weight ($\in[0,1]$).

Assume that I know $a$ and I am able to estimate the empirical cumulative distribution functions (ecdf) of $F$ and $G$ that I note $\hat{F}$ and $\hat{G}$. My question is :

How can I estimate properly $H$ ?

For instance, if I use $\hat{H}(x)=\frac{\hat{F}(x)-a\hat{G}(x)}{1-a}$

then I am not sure that $\hat{H}(x)$ will be an increasing function between 0 and 1. Typically, the expectation of $\hat{H}(x)$ is $H(x)$ but this estimate does not respect the properties of a cumulative distribution function...

My idea was to start from $\hat{H}(x)$ and then estimate a cumulative distribution function from these points. However, I would like to make as few as possible assumptions. Do you know if there is a method for that problem?

Thank you

• It is hard to imagine a setting where you could estimate $G$ separately. Unless you get a separate sample from $G$. Oct 26, 2015 at 16:59
• It is what I get. In fact, I search for a hidden distribution in F, knowing that I know G and a. Oct 26, 2015 at 17:08
• Are your estimates $\hat F$ and $\hat G$ based on independent datasets?
– whuber
Oct 26, 2015 at 17:38
• Yes, they have been estimated independently. Oct 26, 2015 at 17:55
• @Xi'an See the first paragraph of my answer, for an example of this hard to imagine setting :). There's probably only one person in this world who would try to do that!
– air
Oct 26, 2015 at 19:03

Such mixture models are prominently featured in the theory of multiple testing. Here again you have such a mixture; the so-called "two-groups" model. The one group corresponds to hypotheses drawn from the null distribution and the other to hypotheses drawn from the alternative distribution. Indeed, there are people crazy enough to attempt to estimate both $G$ and $H$ and $\alpha$ from the same sample! This is often called empirical null modelling and it was essentially pioneered by Prof. Bradley Efron. One of the assumptions to do that kind of modelling is that $\alpha > 0.9$, but I digress. My main point from this paragraph is that you can find a lot of inspiration to answer your question in the multiple testing literature; and I will attempt to do so below.

What people actually usually end up assuming in the multiple testing literature is that the distribution under the null hypothesis (say it is $G$) is known. Then, using your notation (i.e. $\hat{F}(x)$ is the ECDF), one can estimate $H$ as follows:

$$\hat{H}(x)=\frac{\hat{F}(x)-aG(x)}{1-a}$$

This as you say is unbiased and consistent, but not a distribution function! In one of my favorite preprints, Bodhisattva Sen and Rohit Kumar Patra try to improve the estimation, by imposing exactly the same condition you talked about!

In particular, let $X_1, \dotsc, X_n \sim F$ be the observations from $F$ and assume for now that $\alpha$ is known. Then they solve the following optimization problem:

$$\displaystyle\min_{W \text{ CDF}} \sum_{i=1}^n (W(X_i)-\hat{H}(X_i))^2$$

The argmin of the above is our new estimator $\tilde{H}$ which actually is a distribution function! In other words they project the naive estimator $\hat{H}$ onto the space of distribution functions and thus improve estimation. They show that this is a convex problem which can be quickly solved with PAVA (pool-adjacent-violator algorithm) and then derive lots of nice asymptotic properties of this estimator.

They also go a bit further and show when and how one can also estimate $\alpha$ when it is unknown (it is not always identifiable) and prove results for that as well, though in your case you already know it.

So basically I think you can just apply this method with $G(x)$ replaced by $\hat{G}(x)$. Indeed, because you estimate $G$ based on an independent data-set, I am quite confident that you could also adapt all their asymptotic consistency results to the case where you are also estimating $G$ by its ECDF.

• Thank you very much ! I will read carefully this preprint ! Actually, I was trying to use PAVA to estimate this function, that's odd ! But it was a try to do something, and the beginning of the beginning of their work ! Great minds think alike :) Oct 26, 2015 at 19:16
• Haha you are welcome! Yeah, I agree it is very natural to go for PAVA for this problem! And I wonder how nobody came up with this idea earlier (or maybe someone did :) ). But I guess it might be related to the fact that usually, e.g. for FDR control methods, you just need $G$ (assumed to be known) and $F$, so nobody bothered to look at $H$.
– air
Oct 26, 2015 at 22:12