I am reading this classic paper on convergence properties of EM for Gaussian Mixture Models. In section 5, the authors compare EM with a gradient based inference approach.
The gradient approach requires step-size $\eta$ for parameter update at each iteration $k$ i.e. updation of paramters from $\Theta^k$ to $\Theta^{k+1}$.
The authors claim that for local convergence, one can use Taylor series expansion of the likelihood around the true parameter $\Theta^*$ and claim that the following condition needs to hold true:
$$\left\|\Theta^{(k+1)}-\Theta^{*}\right\| \leq\left\|I+\eta H\left(\Theta^{*}\right)\right\|\left\|\Theta^{(k)}-\Theta^{*}\right\|$$
Here $H$ is the Hessian of the likelihood. I am trying to derive this condition.
Here is my attempt:
Let $L$ be the likelihood. Expanding $L$ around the true parameter $\Theta^*$ gives
$$L(\Theta^k) \approx L(\Theta^*) + \nabla L(\Theta^* - \Theta^k) + 0.5(\Theta^* - \Theta^k)^{T}H(\Theta^* - \Theta^k) $$
Similarly, I expand $L$ for $\Theta^{k+1}$. $$L(\Theta^{k+1}) \approx L(\Theta^*) + \nabla L(\Theta^* - \Theta^{k+1}) + 0.5(\Theta^* - \Theta^{k+1})^{T}H(\Theta^* - \Theta^{k+1}) $$
Also, at $\Theta^*$, the gradient $\nabla L = 0$. Subtracting the above two equations, using the condition to ensure montonocity of $L$ at each iteration, I have
$$ (\Theta^* - \Theta^k)^{T}H(\Theta^* - \Theta^k) - (\Theta^* - \Theta^{k+1})^{T}H(\Theta^* - \Theta^{k+1}) < 0 $$
I am unable to simplify further. Is this the right approach? Thank you in advance!