PCA for probability vectors Is there a procedure equivalent to principal component analysis (PCA) for probability vectors? 
I have an n-by-m array where every column sums to one, and all entries are positive. PCA works in principle, but doesn't really make sense, since it implies that my data comes from a mix of Gaussian distributions. I would like to use something more "honest". 
 A: A proposal for this problem with probability measures can be found
Bigot et al. (2017). Geodesic PCA in the Wasserstein space by convex PCA. In Annales de l'Institut Henri Poincaré, Probabilités et Statistiques.
Given a set of curves $p_1 \ldots p_n \in L_2$ the first functional principal components can be formulated as the problem of finding a sequence of nested affine subspaces minimizing the sum of norms of projection residuals. In particular the first principal component is a solution of
$$ \min_{v \in L_2, \|v\| = 1}  \sum_{i=1}^{n}d^2(p_i, S_v)$$
where $S_v = \{\bar p + tv, t \in \mathbb{R}\}$ and $d(x, S) = \inf_{x \in S} \| x - \bar p\|$.
Assuming $p_1. \ldots. p_n \in  W_2$ the space of probability meassures with the Wasserstein metric, the proposed methodology in the paper is to change the distance defined by the euclidean metric to $d_W$ the Wasserstein metric. The first geodesic component is a solution of
$$ \min_{v \in W}  \sum_{i=1}^{n}d_W^2(p_i, v).$$
The following figures copy pasted from the paper show the different results obtained for usual FPCA and GPCA.  Since the minimizing of the second approach is done over $W$ the results obtained are probability measures. 


A: This sounds like a job for Latent Dirichlet Allocation (LDA), which is a way of modeling probability vectors using low-dimensional latent variables. Here's a link to a tutorial. It starts with matrix factorization, quickly goes through some LDA predecessors, and then explains how matrix factorization relates to LDA. (It also goes on to offer more alternatives that do well on tasks that LDA was not designed for.)
https://www.cs.cmu.edu/~epxing/Class/10708-15/slides/LDA_SC.pdf
A: If you're looking for something that won't make your data entries negative you could try nonnegative matrix factorization.
For an example implementation see scikit-learn's NMF.
A: The problem is actually that of the compositional data analysis (sometimes referred to as CoDa - compositional data), where the data live on a simplex. The key insight here is that the truly meaningful are only the data ratios, and the usual techniques of multivariate statistics become applicable after a logarithmic transformation (with appropriate care), either:

*

*alr - additive log-ratio transformation
or

*clr - centered log-ratio transformation
The latter is defined as
$$
clr(x) = clr(x_1, ..., x_k) =
\left(\log\left(\frac{x_1}{g(x)}\right), ..., \log\left(\frac{x_k}{g(x)}\right)\right),\\
g(x)=\left(\prod_j x_j\right)^\frac{1}{k}.
$$
More details can be found in these and these lecture notes. An important addition is related to treating the zero values, which pose a problem in logarithmic transformation - see here.
