# Questions tagged [information-geometry]

The geometrical analysis of probability spaces

37 questions
Filter by
Sorted by
Tagged with
72 views

### Intuition/meaning of information geometry distances and geodesics?

In information geometry, we consider a manifold of probability distributions, together with the Fisher Information metric (given by the Fisher Information matrix). I have some intuition (see ...
• 1,258
28 views

### Fisher information reparametrization formula in non-bijective cases

The formula writes as this: $$I(X) = \left(\frac{\partial Z}{\partial X}\right)^\top I(Z) \left(\frac{\partial Z}{\partial X}\right),$$ in particular, if $Z = AX$, we have $$I(X) = A^\top I(Z) A.$$ ...
• 101
149 views

### Ay (2017) vs Amari (2016) for information geometry

I am interested in learning information geometry in detail with a focus on applications, and am currently considering two main texts: Amari (2016)'s "Information Geometry and Its Applications&...
1 vote
123 views

### How does computing the bias of model parameters make sense?

I've been studying statistics recently and was thinking about the fact that computing the expectation of a random variable $E(X)$ only really makes sense if $X$ is a random variable defined over a ...
• 153
1 vote
123 views

### How to learn information geometry from a statistical background [closed]

I have started learning information geometry from the book Differential geometry and Statistics by M.K.murray and J.W. Rice.But I am facing huge difficulties in understanding mathematics behind it. I ...
85 views

### Book recommendation: which ones should I study in order to learn information geometry, copulas and everything in between?

So far I have been studying the theory of copulas for a while and I became fascinated by it. More recently, I read about information geometry which also caught my attention. Having said that, could ...
1k views

### Difference of two KL-divergence

The Kullback-Leibler (KL) divergence between two distributions $P$ and $Q$ is defined as $$\mbox{KL}(P \| Q) = \mathbb{E}_P\left[\ln \frac{\mbox{d}P}{\mbox{d}Q}\right].$$ My question is that suppose ...
• 301
283 views

### A divergence that can be extended to logistic functions?

If I have data $\{(x_i, y_i)\}_{i=1}^n$ where the dependent variable is binary $(y_i = 0,1)$ I can model it using a logistic function: $$f(x; \alpha, \beta) = \frac{1}{1 + e^{-(\alpha + \beta x)}}$$ ...
159 views

### Fisher information matrix and gradients

I'm a math Ph.D. without formal training in statistics. Quite a few papers on normalization methods in deep learning mention the Fisher information matrix and how it's related to the Riemannian metric ...
• 171
231 views

### Diffusion tensor as a covariance matrix

TLDR: In nuclear magnetic resonance (NMR), to study molecular diffusion we assume that molecules displace in 3D space according to a trivariate gaussian distribution. The variables are then the ...
63 views

### Is the Franke-Wolfe algorithm the same as Manifold optimization?

The Frank-Wolfe optimization algorithm describes optimization over a constrained domain. In the Manifold Optimization literature (e.g. [1]) a Gradient-Step is done using an exponential map. This maps ...
• 5,684
117 views

### Is MLE intrinsically connected to logs?

My mathematical exploration led me the following claim: Claim: MLE is fundamentally connected to logs (and KL divergence, which also uses logs). It’s not correct to say log shows up simply to make ...
2k views

### How to picture EM algorithm and KL-divergence geometrically?

In reading up on the Expectation-Maximization algorithm on Wikipedia, I read this short and intriguing line, under the subheading "Geometric Intuition": In information geometry, the E step ...
• 191
1k views

### KL Divergence, Bregman, and uniqueness

While reading the following paper on Bregman Divergence (link) Banerjee, Arindam, et al. "Clustering with Bregman divergences." Journal of machine learning research 6.Oct (2005): 1705-1749. In ...
• 1,636
3k views

### What is the most beginner-friendly book for information geometry?

Question: What is the most beginner-friendly book for information geometry? The book: Amari and Nagaoka, Methods of Information Geometry, is often mentioned as a reference for information ...
• 6,259
432 views

### Most accessible introduction to directional statistics?

Question: Does anyone have a recommendation for a reference which is "the most accessible" introduction to directional statistics? When I say "accessible", I mean that many authors are so experienced ...
791 views

### Information geometry tutorial

I would like to know more about the topic of information geometry, but i don't want to delve deeply into it. The resources i found was either too complicated or just very messy presentations like this ...
3k views

### Graphical intuition of statistics on a manifold

On this post, you can read the statement: Models are usually represented by points $\theta$ on a finite dimensional manifold. On Differential Geometry and Statistics by Michael K Murray and ...
• 26.3k
347 views

### Coefficient of variation of correlation values

I have the following problem that I am not sure of whether I treat it correctly: I run repeated simulations, where the output of a single simulation replicate is a single correlation value. The ...
• 41
101 views

### What is Geometrical Probability? [closed]

Is it a technique where Geometry is used to solve probabilistic problems? Is it a kind of probability which grows Geometrically when we conduct experiments? Is it a kind of distribution? I am ...
• 2,088
2k views

### KL divergence vs Absolute Difference between two distributions? [duplicate]

Why should I use KL divergence over just giving the abs difference from two PDFs?
2k views

### Cramer-Rao Lower Bound for the estimation of Pearson correlation

Given a bivariate Gaussian distribution $\mathcal{N}\left(0,\begin{pmatrix} 1 & \rho \\ \rho & 1 \end{pmatrix}\right)$, I am looking for information on the distribution of $\hat{\rho}$ ...
• 4,328
1k views

• 617
578 views

### Orthogonal intersection of linear family and exponential family

I asked the following question in MSE for which I couldn't get any answer yet. I thought this would be a better place for that question. In statistical maniolds $S=\{p_\theta\}$,\$\theta=(\theta_1,\...
• 617
628 views

### References for use of symplectic geometry in statistics?

I have heard that many problems in Mathematical Statistics can be stated and solved in terms of Symplectic Geometry. Unfortunately this was a pretty vague statement and I am interested in something ...
• 153
768 views

### Using information geometry to define distances and volumes…useful?

I came across a large body of literature which advocates using Fisher's Information metric as a natural local metric in the space of probability distributions and then integrating over it to define ...
• 6,209
4k views

### Orthogonal parametrization

In general inference, why orthogonal parameters are useful, and why is it worth trying to find a new parametrization that makes the parameters orthogonal ? I have seen some textbook examples, not so ...
• 549