# How to determine mixture coefficient nonparametrically?

## Problem:

Given a sample $$X_m$$ from each of $$M$$ distributions $$f_m$$ which are all mixtures of the same $$C$$ unknown distributions $$g_c$$ but with differing mixture coefficients $$\alpha_{mc}$$, (when and) how can I determine the mixture coefficients and distributions?

$$X_m \sim \sum_{c=1}^C \alpha_{mc} \cdot g_c(x)\quad$$ where $$\quad\sum_{c=1}^C\alpha_{mc} =1, x \in \mathbf{R}^p \quad\quad \forall m

## What I know (not much):

If I was given only one sample (M=1), this would simply be a clustering problem.

For the case in which x is one dimensional ($$p=1$$) and only two mixtures and two component are concerned (M=C=2) and one sample is from one component alone ($$\alpha_{1,1}=1$$), Patra & Sen (2016) provide bounds on $$\alpha$$ using the ECDF and the monotonicity constraint for the reconstruction of $$G_2(x)$$. However, the curse of dimensionality hits my extension of it to $$p>1$$ hard (ECDF will be 1/Nth most of the time).

I currently approach the problem by maximum likelihood based on the observed k-nearest neighbor (knn) distances. But even with gradient information the optimization (using R's optim) appears difficult (local minima of -log(L)). This optimisation problem is very similar to Non-negative matrix factorization (NMF) objectives and the problem itself similar to hyperspectral unmixing. But I failed applying either directly to my problem.

I have the strong feeling that this problem has been solved before and I just dont't know its name - do you?

• I faced a similar problem, where in my case $M\gg{C}$, the samples $X_m$ were of decent size, and $p=1$. I actually just did gridded kernel-density estimates of the $M$ distributions, and then applied NMF directly. Even in this case the answer was typically under-constrained a bit. If you have large $p$, I think you may need more prior constraint on the form of the end-member distributions. Dec 20 '16 at 2:40
• @GeoMatt22 I actually have M=10, C=2 (at least that is what I think I have), N=5000 each, p=4. As my distributions are pretty smooth, KDE might indeed be a good idea, I guess (I need to read on KDE vs KNNDE). I assume you used KL divergence as loss for NMF? But how did you enforce the "mixture coefficients sum to 1 for each mixture" constrain? I did not find a way to do so in the R package NMF. Dec 21 '16 at 0:47
• My approach was in Matlab and far from rigorous. I used the default Frobenius (matrix-L2) loss. I just interpreted the coefficients as "weights", so normalized by the sum. (The end-member distributions are also not guaranteed to sum to one either. Or rather their "discrete integral", allowing for a cell-volume "$|dX|$" factor.) I had $p=1$, and my "KDE" was actually just histograms post-processed with a Gaussian blur. Dec 21 '16 at 1:11