Questions tagged [information-geometry]

The geometrical analysis of probability spaces

2
votes
0answers
20 views

Fisher information matrix and gradients

I'm a math Ph.D. without formal training in statistics. Quite a few papers on normalization methods in deep learning mention the Fisher information matrix and how it's related to the Riemannian metric ...
1
vote
0answers
13 views

Diffusion tensor as a covariance matrix

TLDR: In nuclear magnetic resonance (NMR), to study molecular diffusion we assume that molecules displace in 3D space according to a trivariate gaussian distribution. The variables are then the ...
1
vote
0answers
40 views

Is the Franke-Wolfe algorithm the same as Manifold optimization?

The Frank-Wolfe optimization algorithm describes optimization over a constrained domain. In the Manifold Optimization literature (e.g. [1]) a Gradient-Step is done using an exponential map. This maps ...
6
votes
1answer
68 views

Is MLE intrinsically connected to logs?

My mathematical exploration led me the following claim: Claim: MLE is fundamentally connected to logs (and KL divergence, which also uses logs). It’s not correct to say log shows up simply to make ...
0
votes
0answers
6 views

Statistical meaning of $\mathbb E_P[score_\theta(Z)]^T\operatorname{Cov}_P[score_\theta(Z)]^{-1}\mathbb E_P[score_\theta(Z)]$

Consider a random vector $Z$ with distribution $P$ having mean $\mu$ and covarance matrix $\Sigma$. Question Statistically what is the meaning of the quadratic quantity $\mu^T\Sigma^{-1}\mu$ ? More ...
4
votes
1answer
336 views

How to picture EM algorithm and KL-divergence geometrically?

In reading up on the Expectation-Mmaximization algorithm on Wikipedia, I read this short and intriguing line, under the subheading "Geometric Intuition": In information geometry, the E step and the ...
3
votes
1answer
575 views

KL Divergence, Bregman, and uniqueness

While reading the following paper on Bregman Divergence (link) Banerjee, Arindam, et al. "Clustering with Bregman divergences." Journal of machine learning research 6.Oct (2005): 1705-1749. In ...
3
votes
1answer
805 views

What is the most beginner-friendly book for information geometry?

Question: What is the most beginner-friendly book for information geometry? The book: Amari and Nagaoka, Methods of Information Geometry, is often mentioned as a reference for information ...
7
votes
1answer
158 views

Most accessible introduction to directional statistics?

Question: Does anyone have a recommendation for a reference which is "the most accessible" introduction to directional statistics? When I say "accessible", I mean that many authors are so experienced ...
6
votes
1answer
304 views

Information geometry tutorial

I would like to know more about the topic of information geometry, but i don't want to delve deeply into it. The resources i found was either too complicated or just very messy presentations like this ...
11
votes
2answers
808 views

Graphical intuition of statistics on a manifold

On this post, you can read the statement: Models are usually represented by points $\theta$ on a finite dimensional manifold. On Differential Geometry and Statistics by Michael K Murray and ...
4
votes
1answer
82 views

Coefficient of variation of correlation values

I have the following problem that I am not sure of whether I treat it correctly: I run repeated simulations, where the output of a single simulation replicate is a single correlation value. The ...
0
votes
1answer
84 views

What is Geometrical Probability? [closed]

Is it a technique where Geometry is used to solve probabilistic problems? Is it a kind of probability which grows Geometrically when we conduct experiments? Is it a kind of distribution? I am ...
4
votes
2answers
450 views

KL divergence vs Absolute Difference between two distributions? [duplicate]

Why should I use KL divergence over just giving the abs difference from two PDFs?
2
votes
2answers
1k views

Cramer-Rao Lower Bound for the estimation of Pearson correlation

Given a bivariate Gaussian distribution $\mathcal{N}\left(0,\begin{pmatrix} 1 & \rho \\ \rho & 1 \end{pmatrix}\right)$, I am looking for information on the distribution of $\hat{\rho}$ ...
14
votes
0answers
537 views

Is there a general expression for ancillary statistics in exponential families?

It is known that an i.i.d sample $X_1,\dots,X_n$ from a scale family with c.d.f. $F(\frac{x}{\sigma})$ has $S(X)$ as an ancillary statistic if $S(X)$ depends on the sample only through $\frac{X_1}{X_n}...
2
votes
0answers
31 views

classification real vs. made-up words

I am interesting in building a classifier that can separates made-up words (such as brands) from real words (belonging to the English dictionary for example). I have tried using a Soundex ...
3
votes
0answers
85 views

A Fisher information metric which doesn't refer to any exponential family

If $\mathcal E$ is some exponential family of distributions, then we can view it as a Riemannian manifold with local metric the Fisher information matrix. We can then define the Fisher metric: the ...
1
vote
0answers
34 views

Request for literature on applications of manifold-valued random variables in medical imaging

I'd appreciate some references/literature on the applications of manifold-valued random variables, i.e., random variables $X:\Omega\to M$, where M is a manifold (could be even infinite dimensional), ...
12
votes
1answer
2k views

Determinant of Fisher information

(I posted a similar question on math.se.) In information geometry, the determinant of the Fisher information matrix is a natural volume form on a statistical manifold, so it has a nice geometrical ...
10
votes
1answer
371 views

Clarification in information geometry

This question is concerned with the paper Differential Geometry of Curved Exponential Families-Curvatures and Information Loss by Amari. The text goes as follows. Let $S^n=\{p_{\theta}\}$ be an $n$-...
4
votes
0answers
870 views

Are there efficient estimators for the variance of an exponential family?

Let us consider the Gaussian model $\mathcal{N}(\mu,\sigma^2)$, where both $\mu$ and $\sigma$ are unknown. I have learnt that (for example, from Amari's information geometry book) the exponential ...
19
votes
3answers
5k views

Does differential geometry have anything to do with statistics?

I am doing master in statistics and I am advised to learn differential geometry. I would be happier to hear about statistical applications for differential geometry since this would make me motivated. ...
6
votes
0answers
238 views

Reference Request: Information Geometry for Ridge Regression

I am reading the book "regression estimators" by Gruber 2010 where he uses this technique to compare Ridge Regressors, however he concentrates on deriving the mathematical results without giving any ...
4
votes
1answer
369 views

Orthogonal intersection of linear family and exponential family

I asked the following question in MSE for which I couldn't get any answer yet. I thought this would be a better place for that question. In statistical maniolds $S=\{p_\theta\}$,$\theta=(\theta_1,\...
4
votes
2answers
271 views

References for use of symplectic geometry in statistics?

I have heard that many problems in Mathematical Statistics can be stated and solved in terms of Symplectic Geometry. Unfortunately this was a pretty vague statement and I am interested in something ...
13
votes
3answers
605 views

Using information geometry to define distances and volumes…useful?

I came across a large body of literature which advocates using Fisher's Information metric as a natural local metric in the space of probability distributions and then integrating over it to define ...
6
votes
2answers
2k views

Orthogonal parametrization

In general inference, why orthogonal parameters are useful, and why is it worth trying to find a new parametrization that makes the parameters orthogonal ? I have seen some textbook examples, not so ...
14
votes
4answers
8k views

Questions about KL divergence?

I am comparing two distributions with KL divergence which returns me a non-standardized number that, according to what I read about this measure, is the amount of information that is required to ...