Need help understanding why the original proof of the EM algorithm had a flaw in it I understand that in the original paper on the EM algorithm by Dempster, the proof of convergence in Theorem 2 had a flaw, and that a full proof wasn't given until CF Jeff Wu published it years later.
I am trying to understand why the proof was flawed. The equations in question are 3.13 to 3.14 in the paper above, where the triangle inequality was supposedly wrongly applied to go from:
$$
\epsilon > \lambda \sum_{j=1}^r \left(\Phi^{p+j}-\Phi^{p+j-1}\right)\left(\Phi^{p+j}-\Phi^{p+j-1}\right)^T
$$
to
$$
\epsilon > \lambda \left(\Phi^{p+r}-\Phi^{p}\right)\left(\Phi^{p+r}-\Phi^{p}\right)^T.
$$
I am failing to see how the triangle inequality was wrongly applied or if there was something deeper here?
 A: This step in the proof seems to be based on the idea that they are using a telescoping sum that reduces in a simple way.  The problem is that the terms in the sum do not actually "telescope" in the way they seem to be assuming.  They seem to be (erroneously) assuming that their sum will "telescope" to give:
$$\begin{align}
\sum_{j=1}^r (\boldsymbol{\Phi}^{(p+j)} - \boldsymbol{\Phi}^{(p+j-1)}) (\boldsymbol{\Phi}^{(p+j)} - \boldsymbol{\Phi}^{(p+j-1)})^\text{T} 
= (\boldsymbol{\Phi}^{(p+r)} - \boldsymbol{\Phi}^{(p)}) (\boldsymbol{\Phi}^{(p+r)} - \boldsymbol{\Phi}^{(p)})^\text{T}. \\[6pt]
\end{align}$$
However, expanding the sum out explicitly actually gives a different result:
$$\begin{align}
\text{SUM} 
&\equiv \sum_{j=1}^r (\boldsymbol{\Phi}_{(p+j)} - \boldsymbol{\Phi}_{(p+j-1)}) (\boldsymbol{\Phi}_{(p+j)} - \boldsymbol{\Phi}_{(p+j-1)})^\text{T} \\[6pt]
&= \sum_{j=1}^r \Big[ 
\boldsymbol{\Phi}^{(p+j)} \boldsymbol{\Phi}^{(p+j) \text{T}} - \boldsymbol{\Phi}^{(p+j)} \boldsymbol{\Phi}^{(p+j-1) \text{T}} - \boldsymbol{\Phi}^{(p+j-1)} \boldsymbol{\Phi}^{(p+j) \text{T}} + \boldsymbol{\Phi}^{(p+j-1)} \boldsymbol{\Phi}^{(p+j-1) \text{T}} \Big] \\[6pt]
&= \quad \Big[ 
\boldsymbol{\Phi}^{(p+1)} \boldsymbol{\Phi}^{(p+1) \text{T}} - \boldsymbol{\Phi}^{(p+1)} \boldsymbol{\Phi}^{(p) \text{T}} - \boldsymbol{\Phi}^{(p)} \boldsymbol{\Phi}^{(p+1) \text{T}} + \boldsymbol{\Phi}^{(p)} \boldsymbol{\Phi}^{(p) \text{T}} \Big] \\[6pt]
&\quad + \Big[ 
\boldsymbol{\Phi}^{(p+2)} \boldsymbol{\Phi}^{(p+2) \text{T}} - \boldsymbol{\Phi}^{(p+2)} \boldsymbol{\Phi}^{(p+1) \text{T}} - \boldsymbol{\Phi}^{(p+1)} \boldsymbol{\Phi}^{(p+2) \text{T}} + \boldsymbol{\Phi}^{(p+1)} \boldsymbol{\Phi}^{(p+1) \text{T}} \Big] \\[6pt]
&\quad + \Big[ 
\boldsymbol{\Phi}^{(p+3)} \boldsymbol{\Phi}^{(p+3) \text{T}} - \boldsymbol{\Phi}^{(p+3)} \boldsymbol{\Phi}^{(p+2) \text{T}} - \boldsymbol{\Phi}^{(p+2)} \boldsymbol{\Phi}^{(p+3) \text{T}} + \boldsymbol{\Phi}^{(p+2)} \boldsymbol{\Phi}^{(p+2) \text{T}} \Big] \\[6pt]
&\quad + \quad \cdots \\[9pt]
&\quad + \Big[ 
\boldsymbol{\Phi}^{(p+r)} \boldsymbol{\Phi}^{(p+r) \text{T}} - \boldsymbol{\Phi}^{(p+r)} \boldsymbol{\Phi}^{(p+r-1) \text{T}} - \boldsymbol{\Phi}^{(p+r-1)} \boldsymbol{\Phi}^{(p+r) \text{T}} + \boldsymbol{\Phi}^{(p+r-1)} \boldsymbol{\Phi}^{(p+r-1) \text{T}} \Big] \\[6pt]
&= \boldsymbol{\Phi}^{(p)} \boldsymbol{\Phi}^{(p) \text{T}} + \boldsymbol{\Phi}^{(p+r)} \boldsymbol{\Phi}^{(p+r) \text{T}} + 2 \sum_{j=1}^{r-1} \boldsymbol{\Phi}^{(p+j)} \boldsymbol{\Phi}^{(p+j) \text{T}} \\
&\quad - 2 \sum_{j=1}^{r} \boldsymbol{\Phi}^{(p+j)} \boldsymbol{\Phi}^{(p+j-1) \text{T}} - 2 \sum_{j=1}^{r} \boldsymbol{\Phi}^{(p+j-1)} \boldsymbol{\Phi}^{(p+j) \text{T}} \\[6pt]
&= (\boldsymbol{\Phi}^{(p+r)} - \boldsymbol{\Phi}^{(p)}) (\boldsymbol{\Phi}^{(p+r)} - \boldsymbol{\Phi}^{(p)})^\text{T} + \boldsymbol{\Phi}^{(p)} \boldsymbol{\Phi}^{(p+r)} + \boldsymbol{\Phi}^{(p+r)} \boldsymbol{\Phi}^{(p)} \\[6pt]
&\quad + 2 \sum_{j=1}^{r-1} \boldsymbol{\Phi}^{(p+j)} \boldsymbol{\Phi}^{(p+j) \text{T}} - 2 \sum_{j=1}^{r} \boldsymbol{\Phi}^{(p+j)} \boldsymbol{\Phi}^{(p+j-1) \text{T}} - 2 \sum_{j=1}^{r} \boldsymbol{\Phi}^{(p+j-1)} \boldsymbol{\Phi}^{(p+j) \text{T}}. \\[6pt]
\end{align}$$
As you can see, this latter result is not generally equivalent to the simpler result that Dempster, Laird and Rubin seem to have used.  I don't think this has anything to do with the triangle inequality --- unless I'm mistaken, I think they just attempted to apply the rules for a telescoping sum and didn't realise that their sum is not of this form.
