Let's assume that we have two samples $\{X_i\}_{i=1..N}$ and $\{Y_i\}_{i=1..M}$ corresponding to random variables $X$ and $Y$. Let there also be a sample $\{Z_i\}_{i=1..K}$ corresponding to random variable $Z$. Assume that $Z$ is unknown mixture of $X$ and $Y$, i.e. $f_Z = \alpha f_X + (1-\alpha) f_Y$.

Is there any way to estimate the alpha coefficient without resorting to parametric hypotheses about $X$ and $Y$? Since we have their samples, it seems possible with the help of kernel density estimation.

P.S. Articles about the nonparametric EM method that I found immediately go into the wilds. But the current case seems to be simple.


1 Answer 1


Maybe I have missed some nuance of your Question; the answer seems almost too simple. Sorry if I'm wasting your time with this.

The brief Wikipedia article on mixture distributions has some relevant material.

Because $f_Z = \alpha f_X + (1-\alpha)f_Y$ you have $$E(Z) = \alpha E(X) + (1-\alpha) E(Y),$$ and then $\alpha = E(Z-Y)/E(X-Y).$

You can use means of the samples you have to estimate the required expectations, and thus $\alpha.$

Here is a brief simulation of 100,000 $Z_i$s that are a mixture with $\alpha = 1/3$ of standard normal and standard uniform. [See this page for more elegant and more general methods of simulation.]

m = 10^5
x = rnorm(m);  y = runif(m)
MAT = cbind(x,y)
id = sample(1:2, m, rep=T, p=c(1,2))
z = numeric(m)
for(i in 1:m) {
 z[i] = MAT[i,id[i]] }
[1] 0.3333354  # aprx 1/3 = 0(1/3) + (1/2)(2/3)
hist(z, prob=T, br=50, col="skyblue2")
 curve((1/3)*dnorm(x)+(2/3)*dunif(x), add=T, col="red", n=10001)

enter image description here

[1] 0.3314798  # aprx 1/3 = alpha
  • 1
    $\begingroup$ This can be generalized to the method of moments (which wil allow you to do a similar trick, but with different expectation values, when E(X) and E(Y) are the same or very close). $\endgroup$ Oct 21, 2020 at 6:54
  • $\begingroup$ @BruceET: What does it mean by $f_{X}$ as well as $f_{Y}$ and $f_{Z}$ ? Do they stand for PDFs of $X$, $Y$ and $Z$ respectively ? IF $Z=\alpha X + (1-\alpha ) Y$, your solution is correct for sure. On the other hand, if $f_{Z} = \alpha f_{X} + (1-\alpha) f_{Y}$, and $f_{X}$, $f_{Y}$ and $f_{Z}$ are PDFs, I am not sure. $\endgroup$
    – user295357
    Oct 21, 2020 at 16:46
  • $\begingroup$ They are PDFs, might have been written as $f_Z(t)=\alpha f_X(t) +(1−\alpha)f_Y(t).$ If you will look at the Wikipedia link, then maybe you will be sure. In what other situation do you suppose this equation would be appropriate? Certainly not for the distribution of $Z = \alpha X + (1-\alpha)Y,$ where $E(Z) = \alpha E(X)+(1-\alpha)E(Y)$ would also be true. Be careful to distinguish between the mixture and the sum of two random variables. // Maybe you want to repeat my simulation, but make a histogram of (1/3)*x+(2/3)*y at the end. $\endgroup$
    – BruceET
    Oct 21, 2020 at 17:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.