7
$\begingroup$

I am putting a von Mises-Fisher prior on my data. The data does lie on a unit sphere, but the only problem is that my data is always positive. So I feel like I am wasting my prior on unnecessary negative space. This problem probably gets exaggerated once I go into high dimensions (~500) since I'm only using a single quadrant out of the possible $2^{500}$.

So my question is how do I work out the normalising constant:

It is known that $\int_{x\in S}\exp(\kappa\mu^Tx)dx$ where S is the surface of the unit sphere is $\frac{(2\pi)^{p/2} I_{p/2-1}(\kappa) }{\kappa^{p/2-1}}$ where $p$ is the number of dimensions and $I$ is the modified Bessel function of the first kind: http://en.wikipedia.org/wiki/Von_Mises%E2%80%93Fisher_distribution.

However, what would happen if I restrict this sphere to the first quadrant. i.e. $x_i>0$?

$$\int_{x\in S, x_i>0}\exp(\kappa\mu^Tx)dx=?$$

$\endgroup$
  • $\begingroup$ I am having a similar problem. Unfortunately, reflecting onto the positive orthant will change the mean of your distributions. An interesting question is whether it changes it the same way that getting rid of non-positive observations does. $\endgroup$ – user80727 Jun 25 '15 at 22:29
  • $\begingroup$ $\mu$ was never the mean to start off with but the mode Reflecting on to the positive orthant should only strengthen the mode right? Anyway, I ended up using the normal vMF and ignored the 'wasted' space on negative area. A Dirichlet prior might be an alternate distribution space to use. $\endgroup$ – sachinruk Jun 26 '15 at 3:46
6
+50
$\begingroup$

Because the analysis should not be too sensitive to the prior, we should feel free to make minor modifications to the prior. Instead of truncating it, why not reflect all the probability into the first hyperquadrant? That is, continue to use a von Mises-Fisher prior for $(x_1,x_2,\ldots,x_{n})$ (with $n\approx 500$) but base your analysis on $(|x_1|,|x_2|,\ldots,|x_{n}|)$. That would not need any renormalization at all.

The objection immediately arises that the calculations would require a $500$-fold sum, amounting to $2^{500}$ terms, which is an impossible calculation. Although that is true, an algebraic simplification makes it possible. I am suggesting using a prior

$$f(\mathbf x; \mu, \kappa) = C(\mu, \kappa) \sum_{i\in \{-1,1\}^n} \exp\left(\kappa (i_1 \mu_1, i_2\mu_2, \ldots, i_n\mu_n) \cdot \mathbf x\right)$$

where $C(\mu,\kappa)$ is the normalizing constant for the von Mises-Fisher distribution with parameters $(\mu, \kappa)$, all the $x_i$ are non-negative (and, without any loss of generality, you may as well assume all the $\mu_i$ are non-negative, too). But by separately performing the sum over the last component, the foregoing can be written

$$C(\mu, \kappa) \sum_{i\in \{-1,1\}^{n-1}} \left(\exp\left(\kappa (i_1 \mu_1, i_2\mu_2, \ldots,\mu_n) \cdot \mathbf x\right) + \exp\left(\kappa (i_1 \mu_1, i_2\mu_2, \ldots,-\mu_n) \cdot \mathbf x\right)\right) \\ = C(\mu, \kappa) 2\cosh(\kappa \mu_n x_n)\sum_{i\in \{-1,1\}^{n-1}} \exp\left(\kappa (i_1 \mu_1, i_2\mu_2, \ldots,i_{n-1}\mu_{n-1}) \cdot \mathbf x_{[-n]}\right)$$

where $\mathbf x_{[-n]} = (x_1, x_2, \ldots, x_{n-1})$. Proceeding inductively on $n$ yields

$$f(\mathbf x; \mu, \kappa) = C(\mu, \kappa) 2^n \prod_{i=1}^n \cosh(\kappa \mu_i x_i)$$

which is quite tractable. For $\kappa \gg 0$ (that is, as this prior grows a little less diffuse), $f(\mathbf x; \mu, \kappa)$ approaches the truncated von Mises-Fisher distribution.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Sorry for the late reply but, how would you find the normalising constant to this? I'm guessing this is not a simple matter of just integrating $cosh(\kappa\mu_i x)$ since I still want to integrate over the surface of the sphere. $\endgroup$ – sachinruk Nov 19 '14 at 4:13
  • $\begingroup$ It's identical to the normalizing constant for the original distribution as given in the question. $\endgroup$ – whuber Nov 19 '14 at 15:21
  • $\begingroup$ How is that possible? You are mirroring the positive quadrants density onto all the other quadrants. Sorry if this is a stupid question. $\endgroup$ – sachinruk Nov 19 '14 at 20:23
  • $\begingroup$ As the post explains, it is putting all the original probability into the first quadrant. Since that does not change the total probability, it will not change the normalizing constant. Specifically, $C(\mu,\kappa)$ is defined in the post as the normalizing constant for the von Mises-Fisher distribution. Its appearance in the final expression for $f(\mathrm{x};\mu,\kappa)$ shows precisely how it is used to normalize $f$. $\endgroup$ – whuber Nov 19 '14 at 20:39
  • $\begingroup$ Ah I see. Bear with me. Lets say $\mu=[1 1]^T$. Thing is your density says that $x=[1 1]^T$ and $x=[-1 -1]^T$ has the same density. This is not the case in the original density. $\mu$ in the original distribution is the mode direction of the density. The density dies down as you move away from it. Therfore I can't see how you can simply reflect it into the first quadrant if it is not symmetric about the origin. $\endgroup$ – sachinruk Nov 19 '14 at 21:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.