Sparsity on the simplex Say I want to minimize a convex function on the probability simplex. How is it possible to encourage the sparsity of the solution (while keeping the problem convex)? 
Since using the sparsity inducing regularization  $l_1$ won't help here 
 A: The Dirichlet distribution can be used to induce sparsity on the simplex. Let $x \in \Delta^{K-1}$ denote that $x$ is a $K$-dimensional vector that lives in the $K-1$ dimensional simplex. Let $\alpha$ denote a $K$-dimensional parameter vector. Then suppose
\begin{equation}
x \sim \textsf{Dirichlet}(\alpha) , 
\end{equation}
where the density is characterized by
\begin{equation}
f(x|\alpha) \propto \prod_{i=1}^K x_i^{\alpha_i-1} .
\end{equation}
The concentration parameter is given by 
\begin{equation}
\alpha_0 = \sum_{i=1}^K \alpha_i . 
\end{equation}
The mean of the distribution is $E[x|\alpha] = \alpha/\alpha_0$. If $\alpha_i = 1$ for all $i$, then the distribution is flat and $\alpha_0 = K$. Sparsity is induced by setting $\alpha_0 < K$. As $\alpha_0$ gets smaller, the probability mass gets more concentrated on the vertices of the simplex. One may interpret this prior for $x$ as encoding the following information: "I think most of the coordinates of $x$ are zero, but I don't know which ones."
