how to derive eq 21.2.3 in BRML, Factor Analysis (Eigen-approach likelihood optimization) how to derive eq 21.2.3 in BRML, chapter21 Factor Analysis?
Log Likelihood function (eq. 21.1.13):
$$
\log{p(\mathcal{V} | \mathbf{F}, \mathbf{\Psi})} = -\frac{N}{2}\left( \mathrm{trace}(\mathbf{\Sigma}_{D}^{-1}\mathbf{S}) + \log{\mathrm{det}(2\pi \mathbf{\Sigma}_{D})} \right)\\
\text{where } \mathbf{\Sigma}_D = \mathbf{F}\mathbf{F}^{T} + \mathbf{\Psi}_{diag} , \mathbf{S} = \frac{1}{N} \sum_{i=1}^{N}(\vec{v}_{i} - \bar{\vec{v}})(\vec{v}_{i} - \bar{\vec{v}})^{T}
$$
eq. 21.2.1 (derivative the log likelihood func with respect to $\mathbf{F}$ and equate to zero, I solved)：
$$
0 = \mathrm{trace}\left( \mathbf{\Sigma}_D^{-1}(\partial_{\mathbf{F}}\mathbf{\Sigma}_{D} )\mathbf{\Sigma}_D^{-1}\mathbf{S} \right) - \mathrm{trace}\left( \mathbf{\Sigma}_D^{-1} \partial_{\mathbf{F}}\mathbf{\Sigma}_{D} \right)\\
$$
eq. 21.2.2 ?:
$$
\partial_{\mathbf{F}} (\mathbf{\Sigma}_{D}) 
= \partial_{\mathbf{F}} (\mathbf{F}\mathbf{F}^{T})
= \mathbf{F} (\partial_{\mathbf{F}} \mathbf{F}^{T}) +  (\partial_{\mathbf{F}} \mathbf{F})\mathbf{F}^{T}
$$
(but, this eq can be derivatived a matrix by matrix ?)
eq. 21.2.3 ?:
$$
\mathbf{\Sigma}_D^{-1} \mathbf{F} = \mathbf{\Sigma}_D^{-1} \mathbf{S}\mathbf{\Sigma}_D^{-1} \mathbf{F}
$$
please tell me this derivation.
 A: I self-solved.
it substitutes eq.21.2.2 for eq.21.2.1 .
$$
\mathrm{trace}\left( \mathbf{\Sigma}_D^{-1}(\mathbf{F} (\partial_{\mathbf{F}} \mathbf{F}^{T}) +  (\partial_{\mathbf{F}} \mathbf{F})\mathbf{F}^{T} )\mathbf{\Sigma}_D^{-1}\mathbf{S} \right) - \mathrm{trace}\left( \mathbf{\Sigma}_D^{-1} (\mathbf{F} (\partial_{\mathbf{F}} \mathbf{F}^{T}) +  (\partial_{\mathbf{F}} \mathbf{F})\mathbf{F}^{T}) \right) = 0\\
\mathrm{trace}\left( \mathbf{\Sigma}_D^{-1} \mathbf{F} (\partial_{\mathbf{F}} \mathbf{F}^{T})\mathbf{\Sigma}_D^{-1}\mathbf{S} +  \mathbf{\Sigma}_D^{-1} (\partial_{\mathbf{F}} \mathbf{F}) \mathbf{F}^{T} \mathbf{\Sigma}_D^{-1}\mathbf{S} \right) 
- \mathrm{trace}\left( \mathbf{\Sigma}_D^{-1} \mathbf{F} (\partial_{\mathbf{F}} \mathbf{F}^{T}) + \mathbf{\Sigma}_D^{-1} (\partial_{\mathbf{F}} \mathbf{F})\mathbf{F}^{T} \right) = 0\\
$$
Using $\mathrm{trace}( \mathbf{A}+\mathbf{B}) = \mathrm{trace} (\mathbf{A})+\mathrm{trace}(\mathbf{B})$, 
$$
\mathrm{trace}\left( \mathbf{\Sigma}_D^{-1} \mathbf{F} (\partial_{\mathbf{F}} \mathbf{F}^{T})\mathbf{\Sigma}_D^{-1}\mathbf{S}\right) 
+ \mathrm{trace}\left(\mathbf{\Sigma}_D^{-1}  (\partial_{\mathbf{F}} \mathbf{F}) \mathbf{F}^{T} \mathbf{\Sigma}_D^{-1}\mathbf{S} \right) 
- \mathrm{trace}\left( \mathbf{\Sigma}_D^{-1} \mathbf{F} (\partial_{\mathbf{F}} \mathbf{F}^{T})\right) 
- \mathrm{trace}\left(\mathbf{\Sigma}_D^{-1} (\partial_{\mathbf{F}} \mathbf{F})\mathbf{F}^{T} \right) = 0\\
$$
reparameterize matrices without effect to derivatives:
$$
\mathbf{A} \equiv \mathbf{\Sigma}_D^{-1} \mathbf{F}, 
\mathbf{B} \equiv \mathbf{\Sigma}_D^{-1}\mathbf{S},
\mathbf{C} \equiv \mathbf{F}^{T} \mathbf{\Sigma}_D^{-1}\mathbf{S},
\mathbf{D} \equiv \mathbf{F}^{T} 
$$
$$
\mathrm{trace}\left( \mathbf{A} (\partial_{\mathbf{F}} \mathbf{F}^{T})\mathbf{B}\right) 
+ \mathrm{trace}\left( \mathbf{\Sigma}_D^{-1} (\partial_{\mathbf{F}} \mathbf{F}) \mathbf{C} \right) 
- \mathrm{trace}\left( \mathbf{A} (\partial_{\mathbf{F}} \mathbf{F}^{T})\right) 
- \mathrm{trace}\left(\mathbf{\Sigma}_D^{-1} (\partial_{\mathbf{F}} \mathbf{F})\mathbf{D} \right) = 0\\
$$
Using $\partial \mathrm{trace}(\mathbf{X}) = \mathrm{trace}(\partial \mathbf{X})$, transpose a derivative operator outside trace.
$$
\partial_{\mathbf{F}}\mathrm{trace}\left( \mathbf{A} \mathbf{F}^{T}\mathbf{B}\right) 
+ \partial_{\mathbf{F}}\mathrm{trace}\left( \mathbf{\Sigma}_D^{-1} \mathbf{F} \mathbf{C} \right) 
- \partial_{\mathbf{F}}\mathrm{trace}\left( \mathbf{A}  \mathbf{F}^{T}\right) 
- \partial_{\mathbf{F}}\mathrm{trace}\left(\mathbf{\Sigma}_D^{-1}  \mathbf{F}\mathbf{D} \right) = \mathbf{0}\\
$$
Using the following derivatives of trace:
$$
\partial_{\mathbf{X}}\mathrm{trace}\left( \mathbf{A} \mathbf{X}^{T}\mathbf{B}\right)=\mathbf{B}\mathbf{A} ,
\partial_{\mathbf{X}}\mathrm{trace}\left( \mathbf{A} \mathbf{X}\mathbf{B}\right)=\mathbf{A}^{T}\mathbf{B}^{T} ,
\partial_{\mathbf{X}}\mathrm{trace}\left( \mathbf{X}\mathbf{A} \right)=\mathbf{A}^{T} ,
\partial_{\mathbf{X}}\mathrm{trace}\left( \mathbf{A}\mathbf{X}^{T} \right)=\mathbf{A} 
$$
So,
\begin{eqnarray}
\mathbf{B}\mathbf{A} 
+ (\mathbf{\Sigma}_D^{-1})^{T} \mathbf{C}^{T}  
- \mathbf{A}  
- (\mathbf{\Sigma}_D^{-1})^{T} \mathbf{D}^{T} 
&=& \mathbf{0} \\
\mathbf{\Sigma}_D^{-1}\mathbf{S}\mathbf{\Sigma}_D^{-1} \mathbf{F} 
+ \mathbf{\Sigma}_D^{-1} (\mathbf{F}^{T} \mathbf{\Sigma}_D^{-1}\mathbf{S})^{T}
- \mathbf{\Sigma}_D^{-1} \mathbf{F} 
- \mathbf{\Sigma}_D^{-1}(\mathbf{F}^{T})^{T}
&=& \mathbf{0}\\
\mathbf{\Sigma}_D^{-1}\mathbf{S}\mathbf{\Sigma}_D^{-1} \mathbf{F} 
+ \mathbf{\Sigma}_D^{-1} \mathbf{S}\mathbf{\Sigma}_D^{-1}\mathbf{F}
- \mathbf{\Sigma}_D^{-1} \mathbf{F} 
- \mathbf{\Sigma}_D^{-1} \mathbf{F}
&=& \mathbf{0}\\ 
2\mathbf{\Sigma}_D^{-1} \mathbf{F}
&=&
2\mathbf{\Sigma}_D^{-1}\mathbf{S}\mathbf{\Sigma}_D^{-1} \mathbf{F}\\
\mathbf{\Sigma}_D^{-1} \mathbf{F}
&=&
\mathbf{\Sigma}_D^{-1}\mathbf{S}\mathbf{\Sigma}_D^{-1} \mathbf{F}\\
\end{eqnarray}
it can derive eq.21.2.3 .
