Let $S$ be the set of all probability distributions on $\mathbb{R}$ and $S_n=\{p_\theta\}$ be an $n$ dimensional submanifold of parameterized family of probability distributions on $\mathbb{R}$ where $\theta=(\theta_1,\dots, \theta_n)\in \Theta$ are called parameters of the probability density $p_\theta(x)$ and $\Theta$ is an open set homeomorphic to $\mathbb{R}^n$. It is also assumed that $p_\theta(x)>0$ for all $x$.

For example, $$p_{\theta=(\mu,\sigma)}(x)=\frac{1}{\sqrt{2\pi}\sigma}e^{\frac{-(x-\mu)^2}{2\sigma^2}}$$ where $\mu\in \mathbb{R},\sigma>0$ forms a $2$-dimensional submanifold.

On this manifold, a Riemannian metric is defined by the following, called Fisher information metric.

\begin{eqnarray*} g_{i,j}(\partial_i,\partial_j) &=&\int \partial_i \log p_\theta(x) ~\partial_j \log p_\theta(x) ~p_\theta(x)~ dx\\ &=& \int \partial_i p_\theta(x) ~\partial_j \log p_\theta(x)~ dx \end{eqnarray*}

where $\partial_i=\frac{\partial}{\partial \theta_i}$.

My question is regarding a point in page 384 from this Ann. Stat. paper by Amari.

In the above mentioned paper, when he says a curve $\{q_t(x)\}$ in $S$, where $q_0(x)=p_\theta(x)$ intersects orthogonally $S_n$ at $q_0(x)=p_\theta(x)$, he means

$$\int \partial_i \log p_\theta(x) ~\frac{d}{dt} \log q_t(x)\lvert_{t=0} ~q_0(x)~dx=0$$. i.e., $$\int \partial_i p_\theta(x) ~\frac{d}{dt} \log q_t(x)\lvert_{t=0} ~dx=0$$.

Why? Could somebody make me understand by giving intuitive explanation of these concepts?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.