Suppose I have many distributions $p_i(\theta)$ I wish to take expectations over $$\mathbb{E}_{p_i}[\mathbf{f}_i(\theta)]$$ where the $\mathbf{f}_i$ are vector-valued. In my problem the $p_i$ share some structure and this lets me find $$\theta^*_i = \textrm{argmax}_\theta\ p_i(\theta)$$ for all $i$ very efficiently. My $p_i$ are multimodal and sparse in that $$\log p_i(\theta_1) \gg \log p_i(\theta_0)$$ if $\theta_1$ is near a mode and $\theta_0$ isn't. Also I can find the locations of the top $M$ modes of the $p_i$ efficiently. I should also say $\theta$ lives in a discrete space with a spatial structure which in my case are indexes of pixels in an image.

As the $\mathbf{f}_i$ are vector-valued I can't try to choose importance distributions close to the optimal $$q_i(\theta) \propto |\mathbf{f}_i(\theta) - \mu| p_i(\theta)$$ I'm trying to choose $q_i(\theta) \propto p_i(\theta)$ with fatter tails.

My question is whether this is a well-studied problem and if so can you point me to any relevant literature? Are there any examples of the use of MLE estimates to design importance distributions I can cite?

I'm also wondering once I have identified a pixel in a mode for a $p_i$ should I try to find the edges of the mode through thresholding $p_i$ or is finding the curvature of $p_i$ at the mode peak and dropping a t distribution on it a more popular/better technique? Perhaps this depends very much on $p_i$.

  • $\begingroup$ Yes, this is rather common. $\endgroup$ – Xi'an May 22 '19 at 19:42
  • $\begingroup$ Thanks for the comment! I will try to make it a little more concrete. $\endgroup$ – Epimetheus May 23 '19 at 8:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.