What to chose for prior and proposal function for MCMC of a mixture

Question

I generated some data according to a mixture of two lognormals: $f(x) = p \cdot \mathcal LN(\mu_1, \sigma) + (1-p) \cdot \mathcal LN (\mu_2, \sigma)$ Now I want to use MCMC to find the parameters $p, \mu_1, \mu_2, \sigma$ but I'm unsure what to chose as the prior and the proposal function.

For the prior I chose:

$p \sim Beta$ but what hyperparameters are best?

$\mu_1, \mu_2 \sim Normal$, again, what hyperparameters?

$\sigma \sim Inverse~Gamma$; hyperparameters?

the proposal is a bit more difficult because $p$ has to be drawn of the $[0,1]$ interval. So

$p \sim truncated~Normal$ in $[0,1]$; does this make sense?

$\mu_1, \mu_2, \sigma \sim Normal$ but how do I avoid negative $\sigma$ ?

Is there an R MCMC package that allows custom proposal functions?

Answer 1

what hyperparameters are best?

How to select priors is a question that is far too broad for this forum, is somewhat subjective, and entirely depends on what kind of problem you're trying to solve, and what an acceptable prior means to you, for your problem. No one else can do your analysis for you.

For example, one might characterize priors as noninformative, weakly informative and strongly informative. Within each of these taxonomies, reasonable people may disagree about which specific prior fits in each category. And you might disagree about which of those three paradigms are most appropriate.

p∼truncatedNormal in [0,1]; does this make sense?

But what happens when your mean is negative and your standard deviation is small, i.e. when most of the probability mass falls below 0? Rejection sampling will reject almost all of your samples, so the sampler will be slow.

An obvious choice for a parameter in the unit interval is the beta distribution.

Alternatively, you could transform the normal draw to lie in the unit interval: define some function $\mathbb{R}\to[0,1]$.

how do I avoid negative $\sigma$ ?

One of several ways. You could estimate $z=\log (\sigma)$, so that $z \in \mathbb{R}$ and any model of $z$ with real support is acceptable. Or you could select a distribution that only has positive support, e.g. the gamma distribution.

Is there an R MCMC package that allows custom proposal functions?

Yes; in rstan you have the option of penalizing the log-likelihood by any arbitrary function. It includes many default options, but you can also write your own.