3
$\begingroup$

I was looking at the probability density function of the multivariate normal distribution and that of the inverse-Wishart distribution:

$$ \begin{array}{rccll} p_{\mathcal{N}}\left(\mathbf{x}; \mathbf \mu, \mathbf \Sigma \right) = & \frac{1}{ {2}^{\frac{p}{2}} \pi^{\frac{p}{2}} } & & \left|\mathbf{\Sigma}^{-1}\right|^{\frac{1}{2}} & e^{-\frac{1}{2}\operatorname{tr}({ {\left( \mathbf{x} - \mathbf{\mu} \right)} {\left( \mathbf{x} - \mathbf{\mu} \right)}^{\mathsf T} }\mathbf{\Sigma}^{-1})} \\ p_{\mathcal{W}^{-1}}\left(\mathbf{\Sigma}; \nu, \mathbf \Psi \right) = & \frac{1}{2^{\frac{\nu p}{2}}\Gamma_p(\frac{\nu}{2})} & \left|{\mathbf\Psi}\right|^{\frac{\nu}{2}} & \left|\mathbf{\Sigma}^{-1}\right|^{\frac{\nu+p+1}{2}} & e^{-\frac{1}{2}\operatorname{tr}({\mathbf\Psi}\mathbf{\Sigma}^{-1})} \end{array} $$ where $p$ is the dimension of the positive-definite matrix $\mathbf \Sigma$.

The uncanny resemblance between the two had me wonder whether the normalization constant in $p_{\mathcal{N}}\left(\mathbf{x}; \mathbf \mu, \mathbf \Sigma \right)$, i.e. ${2}^{\frac{p}{2}} \pi^{\frac{p}{2}}$, can be somewhat related to that in $p_{\mathcal{W}^{-1}}\left(\mathbf{\Sigma}; \nu, \mathbf \Psi \right)$, i.e. $2^{\frac{\nu p}{2}}\Gamma_p(\frac{\nu}{2})$.

My question is: is ${2}^{\frac{p}{2}} \pi^{\frac{p}{2}}$ a special case of $2^{\frac{\nu p}{2}}\Gamma_p(\frac{\nu}{2})$ when $\nu$ takes a specific value? If so, how to prove it?

I tried to equate the two and solve for $\nu$ but was not particularly successful. The most promising lead I had was to expand $\Gamma_p(\frac{\nu}{2})$ using the one of the properties of multivariate gamma function listed in Wikipedia: $$ \Gamma _{p}(a)=\pi ^{{p(p-1)/4}}\prod _{{j=1}}^{p}\Gamma \left[a+(1-j)/2\right]. $$ or maybe I had been pursing the wrong direction entirely?

$\endgroup$
2
$\begingroup$

I’m going to reparameterize $a=\nu/2$.

If $p=1$ then we can take $\nu = 1 \implies a = \frac 12$ to get $$ 2^{\nu p/2}\Gamma_{p}(\nu/2) = \sqrt{2\pi} = (2\pi)^{p/2} $$ so there does exist a $\nu$ such the two are equal in the trivial case of $p=1$.


In general, though, we have the following result:

Claim: for $p \in \{2, 3, \dots\}$ we have $$ 2^{ap} \Gamma_p(a) > (2\pi)^{p/2} $$ for all $a>(p-1)/2$.

Proof: by induction on $p$.

Base case: $p=2$. I claim that $$ f(a) := 2^{2a} \Gamma_2(a) - 2\pi > 0 $$ for $a>1/2$. We have the identity that $$ \Gamma_2(a) = \pi^{1/2} \Gamma(a)\Gamma(a - 1/2) $$ so $f(a) = 2^{2a}\pi^{1/2}\Gamma(a)\Gamma(a-1/2) - 2\pi$. Let $h(a) = 2^{2a}\pi^{1/2}\Gamma(a)\Gamma(a-1/2)$. Recalling that $\log \Gamma$ is convex, this means $$ \log h(a) = a\log 4 + \frac 12 \log \pi + \log \Gamma(a) + \log \Gamma(a-1/2) $$ is too, and $x\mapsto e^x$ is convex and non-decreasing, so we know $\exp \circ \log \circ h = h$ is convex. The function $x \mapsto -2\pi$ is also convex so this means that $f$ itself is convex.

I'm going to use the convexity of $f$ to construct a piecewise linear function $g$ such that $f \geq g > 0$. The linear pieces of $g$ will be supporting hyperplanes (ie tangent lines) of $f$. Letting $h_x(a)$ be the tangent line of $f$ at $x$, we have $h_x(a) = f(x) + f'(x)(a - x)$. By the convexity of $f$ I know that for all $x \in (1/2, \infty)$, $f \geq h_{x}$. Numerically you can find that the minimum of $f$ is at $a_0 \approx 1.23$, so I'm going to obtain $h_{1}$ and $h_{1.5}$ and then set $g(a) := \max\{h_1(a), h_{1.5}(a)\}$. It's important that $1 < a_0 < 1.5$ because this tells me that $h_1$ is decreasing while $h_{1.5}$ is increasing, so the minimum of $g$ is at their intersection. If I can show this is positive then I'm done.

In order to work out $h_1$ and $h_{1.5}$ we need $f'$, which using the product rule and substituting $\psi = \Gamma' / \Gamma$ we have $$ f'(a) = 2^{2a}\Gamma_2(a)\left(\log 4 + \psi(a) + \psi(a-1/2)\right) $$ where $\psi$ is the digamma function. It is well-known that $\Gamma(z+1) = z\Gamma(z)$ and $\psi(1 + z) = \frac 1z + \psi(z)$. We also have the special values $\Gamma(1/2) = \sqrt \pi$, $\psi(1) = -\gamma$, and $\psi(1/2) = -2\log 2 - \gamma$ where $\gamma$ is the Euler–Mascheroni constant. All together we can use those special values and the aforementioned identities to work out $$ h_1(a) = 2\pi - 8\pi \gamma (a - 1) $$ and $$ h_{1.5}(a) = 2\pi + 8\pi(1-\gamma)(a - 1.5). $$ These two lines intersect at $\alpha := \frac{3-\gamma}{2}$, and at that point we have $$ g(\alpha) = 2\pi - 8\pi\gamma \left(\frac{3-\gamma}{2} - 1\right) > 0. $$

This means that for all $a \in (1/2, \infty)$ we have $f(a) \geq g(a) > 0$ and that concludes the base case.


Inductive case: suppose true for $p=2,3,\dots, n$. Then in particular we have $$ \Gamma_n(a) > \pi^{n/2}2^{n(1/2-a)} $$ for all $a>(n-1)/2$ and we want to show $$ \Gamma_{n+1}(a) \stackrel{\text{NTS}}> \pi^{(n+1)/2}2^{(n+1)(1/2-a)} $$ for all $a>n/2$. Note that $1/2-a<0$ so $2^{1/2-a} < 1$. I'm going to use that $\stackrel{\text{NTS}}>$ (NTS = Need To Show) to denote inequalities that we are trying to prove but haven't yet.

From the linked Wikipedia article on the multivariate Gamma function we have $$ \Gamma_{n+1}(a)=\pi^{n/2}\Gamma_n(a)\Gamma(a - n/2) $$ $$ > \pi^n 2^{n(1/2-a)} \Gamma(a-n/2) $$ $$ > \pi^n 2^{(n+1)(1/2-a)} \Gamma(a-n/2) $$ so if we show $$ \pi^n \Gamma(a-n/2) \stackrel{\text{NTS}}> \pi^{(n+1)/2} $$ then we’re done.

This is equivalent to showing that $$ \Gamma(a-n/2) \stackrel{\text{NTS}}> \pi^{(1-n)/2}. $$ $a-n/2>0$ so the smallest that $\Gamma(a-n/2)$ can be is its unique minimum on the positive reals which is greater than $0.75$. But for $n\geq 3$ we have $\pi^{(1-n)/2}<1/3$ so no matter what for any valid $a$ we have $\Gamma(a-n/2) > \pi^{(1-n)/2}$.

Putting this all together, we have established the following chain of inequalities: $$ \pi^{(n+1)/2}2^{(n+1)(1/2-a)} = \pi^{(n+1)/2}2^{n(1/2-a)}2^{1/2-a} $$ $$ < \pi^{(n+1)/2}2^{n(1/2-a)} < \pi^n 2^{n(1/2-a)}\Gamma(a - n/2) $$ $$ = \pi^{n/2} \left[\pi^{n/2} 2^{n(1/2-a)}\right]\Gamma(a - n/2) < \pi^{n/2} \Gamma_{n}(a) \Gamma(a - n/2) $$ $$ = \Gamma_{n+1}(a) \implies 2^{a(n+1)}\Gamma_{n+1}(a) > (2\pi)^{(n+1)/2} $$ for all $a > ((n+1)-1)/2$.

$\square$

$\endgroup$
  • $\begingroup$ This is a great answer! I think I will probably have to digest it for a bit more though. I'd thought the normalization constant in multivariate normal would possess some intuitive link to that of its conjugate prior since the d.f. parameter $\nu$ often have the intuitive interpretation of the number of observations made before seeing new data. With the Bernoulli dist., for example, the normalization constant can be thought of as $\mathrm{B}\left(1, 0\right)$ or $\mathrm{B}\left(0, 1\right)$; same can be said for univariate normal distribution as suggested by the trivial case $p = 1$. $\endgroup$ – Xiubo Zhang Mar 29 '18 at 10:48
  • $\begingroup$ @XiuboZhang Thanks! I will think some more about the intuitive link too and will update if I come up with anything worth sharing. I also just changed the base case considerably to avoid relying on a numerical estimate of the root of the derivative (that root still appears but the exact value isn't an issue any more) $\endgroup$ – jld Mar 30 '18 at 17:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.