Bayesian Regression- Expectation Maximization In Bayesian regression, we have $y_i=x_i^{T}w+\epsilon_i$ where $w \sim \mathcal{N}(0,\alpha)$ and $\epsilon_i \sim \mathcal{N}(0,\frac{1}{\beta})$.
Inference of $\alpha$ and $\beta$ is done by maximizing the likelihood(or marginal likelihood) given the data. This paper(Appendix A.1) explains that maximization can be done by expectation-maximization(EM).
My questions is that why do we need EM to do the maximization. I understand that EM can be used by treating the $w$ as the hidden variable. However, $w$ can be integrated out and we can apply gradient descent(GD). Why don't we use GD?
 A: You ask "why do we need EM to do the maximization", but the author says "Another strategy to maximize (36) is to exploit an EM formulation". Clearly, we don't need EM. However, it is very interesting content for an appendix, since under some circumstances EM might come in handy. For example, gradient descent is not guaranteed to increase the likelihood in each iteration, while EM is. Also, although not in this problem, sometimes you need to estimate parameters that must fulfill a constraint (e.g. the sum of the weights must add to one) and EM can include such constraints in a straightforward manner while gradient descent can step into regions where the constraint is not fulfilled. Moreover, EM algorithms tend to be numerically more stable than gradient based ones. In my opinion, EM can be a superior alternative to gradient-based methods, depending on the specific details of your problem.
UPDATE
In this book, section 9.3.4 the same problem is solved using both methods. However, the author does not address the superiority of any of them over the other which suggests that, at least in this case, there's no obvious superiority of any. Even more, he claims that "these re-estimation equations are formally equivalent to those obtained by direct
maximization."
