Algebraic Geometry for Statistics

Question

I have heard about uses of Algebraic Geometry in Statistics and Machine Learning. I wanted to try to learn a bit about this topics. I don't know nearly anything about Algebraic Geometry, but I have background in math, and I know about basic group theory, rings fields and some commutative algebra. My questions are:

1. What are the Algebriac Geometric concepts I should learn that are connected to applications in Stats/ML (I suppose only a portion of what is usually taught in Algebraic Geometric courses and books is useful).

2. Can you recommend some books / introductory papers for someone with my background? I don't mean standard textbooks for AG but something that focuses on concepts used in applications.

Answer 1

Here is a list of the standard references:

• Drton, Sturmfels, and Sullivant, Lectures on Algebraic Statistics
• Pachter, Sturmfels, Algebraic Statistics for Computational Biology
• Pistone, Riccomagno, Wynn, Algebraic Statistics: Computational Commutative Algebra in Statistics
• Sullivant, Algebraic Statistics
• Zwiernik, Semialgebraic Statistics and Latent Tree Models
• Riccomagno, A Short History of Algebraic Statistics

Here is a list of related references, not directly addressing algebraic statistics, although providing background in the methodology used for the subject:

• Lauritzen, Graphical Models
• Lauritzen, Concrete Abstract Algebra
• Pearl, Causality: Models, Reasoning, and Inference
• Cox, Little, O'Shea, Ideals, Varieties, and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra
• Garrity, Belshoff, Boos, et al, Algebraic Geometry: A Problem-Solving Approach
• Sturmfels, Solving Systems of Polynomial Equations
• Uhler, Geometry of Maximum Likelihood Estimation in Graphical Models

Web pages of courses about the topic, past and present:

• Professor Sturmfels's course at Berkeley
• Course at the TU Berlin
• Seminar at the Freie Universität, Berlin
• Conference at the University of Genoa
• Berkeley Algebraic Statistics Seminar

These lists are almost certainly by no means comprehensive.

Answer 2

• Sumio Watanabe, Algebraic Geometry and Statistical Learning Theory, Cambridge University Press, Cambridge, UK, 2009.

Sure to be influential, this book lays the foundations for the use of algebraic geometry in statistical learning theory. Many widely used statistical models and learning machines applied to information science have a parameter space that is singular: mixture models, neural networks, HMMs, Bayesian networks, and stochastic context-free grammars are major examples. Algebraic geometry and singularity theory provide the necessary tools for studying such non-smooth models. Four main formulas are established:

1. the log likelihood function can be given a common standard form using resolution of singularities, even applied to more complex models;

2. the asymptotic behaviour of the marginal likelihood or 'the evidence' is derived based on zeta function theory;

3. new methods are derived to estimate the generalization errors in Bayes and Gibbs estimations from training errors;

4. the generalization errors of maximum likelihood and a posteriori methods are clarified by empirical process theory on algebraic varieties.