The question is that I have several probability distributions. For each distribution $P_i$, I don't know what it is. Instead, I have a set of observations $\{x_i^j\}_{j=1}^{n_i}$ drawn from each of the $P_i$. Now for a new observation $x_t$, is there a way to estimate the likelihood of each $P_i$ the new observation belongs to? Or which $P_i$ it most likely belongs to? Here $x_i^j, x_t \in R^m$.

I do hope for a method that can directly calculate the likelihoods without making assumptions on the distributions and calculating their parameters first. But honestly I don't know if such methods even exist.


  • $\begingroup$ Have you looked at classification algorithms. For example k nearest neighbors. $\endgroup$
    – shane
    Jul 3, 2015 at 7:52
  • $\begingroup$ @shane Thanks! I didn't even realize that it was a classification problem! In fact the observations $\{x_i^j\}_{j=1}^{n_i}$ are labeled by $i$, so it is a supervised learning problem. $\endgroup$
    – yayaha
    Jul 3, 2015 at 15:57

1 Answer 1


If I understand your problem correctly, I doubt that it is solvable in the general sense.

In fact (if I do understand your problem), you can easily construct a trivial example which has an infinite number of non-unique sets of $P_i$ from which the $x_i$ could have come from.

For example: suppose you have two hidden discrete RVs such that $P_1 = 1$ with probability 1 and $P_2 = 2$ with probabilty 1. Assume also, that the probability you draw an $x_i$ from either of the two distributions is $\frac{1}{2}$ and that these drawings are indept.

You could never distinguish this scenario from one where the distributions where defined such that: $P_1(X=1)=0.25$ and $P_1(X=2)=0.75$, whilst $P_2(X=1)=0.75$ and $P_2(X=2)=0.25$ and the $x_i$ were independently drawn from $P_1$ and $P_2$ with probability $\frac{1}{2}$.

  • $\begingroup$ Hi Jonathan, thanks for your answer! Maybe I didn't state my question well, so sorry if I have confused you. @shane just reminded me that my question, in essence, is a classification problem. I have a set of observations $\{x_i^j\}_{j=1}^{n_i}$ drawn from each distribution (class) $P_i$, and I want to know the probability that a new observation belongs to each of the distribution (class). I now have realized it could be solved by training a classifier. $\endgroup$
    – yayaha
    Jul 3, 2015 at 15:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.