EM algorithm with constraints Lets say that we want to constraint some of the variables in a model, we denote the model's parameters by $\theta \in \mathbb{R}^n$ and we want to train the model on input data $X$.
Normally we would just use EM, but lets say that instead of $\theta \in \mathbb{R}^n$ we have $\theta \in U \subset \mathbb{R}^n$ where $U$ is known. For example, $\theta \in \{(a, a, b)| a + a + b = 1\} \subset \{(a, b, c)\}$.
Intuitively, and by some examples I did, I would use EM iteration with an additional step where we fix the parameters to be in $U$. Something similar to this:

Where now $y_{t + 1}$ is the output after the EM iteration with $\theta^{(t)} = x_t$ instead of whats written in the picture and $x_{t+1}$ as in the picture.
Is there some known results/algorithms on this matter?
 A: Projection onto the feasible set is one of the standard methods of handling constraint violations in convex optimization.  There are other approaches, e.g., stepping only partway to the boundary of the feasible set, or adding a penalty for the amount of violation of the feasible set (which reduces how much violation you'll have), or adding a penalty for how close you get to the boundary of the feasible set that goes to infinity at the boundary / limit of the set.  If your gradient descent algorithm is performing a line search / approximate line search for the minimizing point along the gradient, all of these methods work (including projection), but if you are adopting a fixed step size (that decreases at each iteration) then you are more limited, and projection becomes the algorithm of choice.
Edit in response to comments:
With respect to the projection operator and linear or nonlinear equality constraints (as in your example) on the parameter spaces, the following paper Constrained EM algorithm with projection method in Computational Statistics (2012) addresses exactly this issue and shows both convergence and monotonic increases in the likelihood function.  With respect to linear inequality constraints, On algorithms for restricted maximum likelihood estimation in Computational Statistics and Data Analysis (2004) proposes a globally convergent projection EM algorithm.
