I have a Dirichlet distribution over the parameters of a multinomial, and I want to estimate its posterior density given some set of evidence. Due to some pecularities of my problem (e.g. presence of "soft evidence", among others), I unfortunately cannot use a simple update counts on the parameters, but I have to resort to sampling techniques.
Once I have collected enough samples for the posterior distribution, I then need to re-estimate its density. Note that I don't necessarily need to estimate a Dirichlet again, the only thing I need is a probability density function. I thought of two possible strategies, one parametric and one non-parametric:
- Try to re-estimate the Dirichlet parameters (alpha counts) from the samples. It seems theoretically possible (see here for instance), but quite difficult to implement, since there is no closed-form solution.
- Alternatively, I could give up trying to re-estimate the Dirichlet counts entirely, and resort to kernel density estimations based on my samples.
I'm currently trying to implement the second strategy, based on a kernel density estimation with Gaussian kernels (see e.g. here). I unfortunately don't seem to get the right result, the calculated densities being too high.
As I understand it, the problem might be that the samples of my Dirichlet are not standard multivariate values: namely, they are constrained to be between 0 and 1 and sum up to 1. They are therefore very strongly correlated. For instance, for a Dirichlet with 2 dimensions (aka a Beta distribution), if the first variable has a value 0.6, the second variable is constrained to have a value 0.4. All the other values have a null density.
I am therefore wondering if it is possible to perform a multivariate kernel density estimation for values drawn from a Dirichlet distribution. If yes, how do you think I should do it? Should I use another kernel? If no, do you have any alternative to KDE to suggest?