I have a set of probability distributions. I am also going to be sampling some data and I would like to determine which distribution the data fits to best. But there are some caveats:
- The probability distributions are beliefs moreso than probability distributions.
- I will have very small samples. This means that I would like to fit the sample to the distribution (or belief) after a minimal number of samples. I would love for it to happen in real time i.e. to fit the sample to the distribution after 1 sample, then 2 samples, then 3 samples etc. and then at some point to say that this sample is definitely from this distribution/belief
- The distributions/beliefs are discrete. In fact, I would describe the distributions/beliefs as being of the family of the multinomial distribution
- There is also the probability that the distribution/belief which the sample is best suited to may change after N samples, where N is random. So as an example, lets assume that the data is best suited to distribution/belief number 1, and that there are N samples: at first we are uncertain which distribution the samples falling into, but then after drawing a few (say, k samples, where k
Given the above, I am considering a few options for how to do this:
- Maximum likelihood expectations seem like a natural way forward
- I also hear that the EM algorithm is great for problems of this kind i.e. when the probability distributions are uncertain and can best be described as a belief
My questions are then:
- What would be the best way to fit the data to one of the distributions/beliefs? Is the EM algorithm indeed better for problems of this nature? Or is there a better way?
- List item