Questions tagged [information-geometry]

The geometrical analysis of probability spaces

29 questions
20 views

I'm a math Ph.D. without formal training in statistics. Quite a few papers on normalization methods in deep learning mention the Fisher information matrix and how it's related to the Riemannian metric ...
13 views

Diffusion tensor as a covariance matrix

TLDR: In nuclear magnetic resonance (NMR), to study molecular diffusion we assume that molecules displace in 3D space according to a trivariate gaussian distribution. The variables are then the ...
40 views

Is the Franke-Wolfe algorithm the same as Manifold optimization?

The Frank-Wolfe optimization algorithm describes optimization over a constrained domain. In the Manifold Optimization literature (e.g. ) a Gradient-Step is done using an exponential map. This maps ...
68 views

Is MLE intrinsically connected to logs?

My mathematical exploration led me the following claim: Claim: MLE is fundamentally connected to logs (and KL divergence, which also uses logs). It’s not correct to say log shows up simply to make ...
6 views

Statistical meaning of $\mathbb E_P[score_\theta(Z)]^T\operatorname{Cov}_P[score_\theta(Z)]^{-1}\mathbb E_P[score_\theta(Z)]$

Consider a random vector $Z$ with distribution $P$ having mean $\mu$ and covarance matrix $\Sigma$. Question Statistically what is the meaning of the quadratic quantity $\mu^T\Sigma^{-1}\mu$ ? More ...
336 views

How to picture EM algorithm and KL-divergence geometrically?

In reading up on the Expectation-Mmaximization algorithm on Wikipedia, I read this short and intriguing line, under the subheading "Geometric Intuition": In information geometry, the E step and the ...
575 views

KL Divergence, Bregman, and uniqueness

While reading the following paper on Bregman Divergence (link) Banerjee, Arindam, et al. "Clustering with Bregman divergences." Journal of machine learning research 6.Oct (2005): 1705-1749. In ...
805 views

What is the most beginner-friendly book for information geometry?

Question: What is the most beginner-friendly book for information geometry? The book: Amari and Nagaoka, Methods of Information Geometry, is often mentioned as a reference for information ...
158 views

Most accessible introduction to directional statistics?

Question: Does anyone have a recommendation for a reference which is "the most accessible" introduction to directional statistics? When I say "accessible", I mean that many authors are so experienced ...
304 views

Information geometry tutorial

I would like to know more about the topic of information geometry, but i don't want to delve deeply into it. The resources i found was either too complicated or just very messy presentations like this ...
808 views

Graphical intuition of statistics on a manifold

On this post, you can read the statement: Models are usually represented by points $\theta$ on a finite dimensional manifold. On Differential Geometry and Statistics by Michael K Murray and ...
82 views

Coefficient of variation of correlation values

I have the following problem that I am not sure of whether I treat it correctly: I run repeated simulations, where the output of a single simulation replicate is a single correlation value. The ...
84 views

What is Geometrical Probability? [closed]

Is it a technique where Geometry is used to solve probabilistic problems? Is it a kind of probability which grows Geometrically when we conduct experiments? Is it a kind of distribution? I am ...
450 views

KL divergence vs Absolute Difference between two distributions? [duplicate]

Why should I use KL divergence over just giving the abs difference from two PDFs?
1k views

Cramer-Rao Lower Bound for the estimation of Pearson correlation

Given a bivariate Gaussian distribution $\mathcal{N}\left(0,\begin{pmatrix} 1 & \rho \\ \rho & 1 \end{pmatrix}\right)$, I am looking for information on the distribution of $\hat{\rho}$ ...
537 views

271 views

References for use of symplectic geometry in statistics?

I have heard that many problems in Mathematical Statistics can be stated and solved in terms of Symplectic Geometry. Unfortunately this was a pretty vague statement and I am interested in something ...
605 views

Using information geometry to define distances and volumes…useful?

I came across a large body of literature which advocates using Fisher's Information metric as a natural local metric in the space of probability distributions and then integrating over it to define ...