2
$\begingroup$

how to derive eq 21.2.3 in BRML, chapter21 Factor Analysis?

Log Likelihood function (eq. 21.1.13):

$$ \log{p(\mathcal{V} | \mathbf{F}, \mathbf{\Psi})} = -\frac{N}{2}\left( \mathrm{trace}(\mathbf{\Sigma}_{D}^{-1}\mathbf{S}) + \log{\mathrm{det}(2\pi \mathbf{\Sigma}_{D})} \right)\\ \text{where } \mathbf{\Sigma}_D = \mathbf{F}\mathbf{F}^{T} + \mathbf{\Psi}_{diag} , \mathbf{S} = \frac{1}{N} \sum_{i=1}^{N}(\vec{v}_{i} - \bar{\vec{v}})(\vec{v}_{i} - \bar{\vec{v}})^{T} $$

eq. 21.2.1 (derivative the log likelihood func with respect to $\mathbf{F}$ and equate to zero, I solved):

$$ 0 = \mathrm{trace}\left( \mathbf{\Sigma}_D^{-1}(\partial_{\mathbf{F}}\mathbf{\Sigma}_{D} )\mathbf{\Sigma}_D^{-1}\mathbf{S} \right) - \mathrm{trace}\left( \mathbf{\Sigma}_D^{-1} \partial_{\mathbf{F}}\mathbf{\Sigma}_{D} \right)\\ $$

eq. 21.2.2 ?:

$$ \partial_{\mathbf{F}} (\mathbf{\Sigma}_{D}) = \partial_{\mathbf{F}} (\mathbf{F}\mathbf{F}^{T}) = \mathbf{F} (\partial_{\mathbf{F}} \mathbf{F}^{T}) + (\partial_{\mathbf{F}} \mathbf{F})\mathbf{F}^{T} $$

(but, this eq can be derivatived a matrix by matrix ?)

eq. 21.2.3 ?:

$$ \mathbf{\Sigma}_D^{-1} \mathbf{F} = \mathbf{\Sigma}_D^{-1} \mathbf{S}\mathbf{\Sigma}_D^{-1} \mathbf{F} $$

please tell me this derivation.

$\endgroup$
1
$\begingroup$

I self-solved.

it substitutes eq.21.2.2 for eq.21.2.1 .

$$ \mathrm{trace}\left( \mathbf{\Sigma}_D^{-1}(\mathbf{F} (\partial_{\mathbf{F}} \mathbf{F}^{T}) + (\partial_{\mathbf{F}} \mathbf{F})\mathbf{F}^{T} )\mathbf{\Sigma}_D^{-1}\mathbf{S} \right) - \mathrm{trace}\left( \mathbf{\Sigma}_D^{-1} (\mathbf{F} (\partial_{\mathbf{F}} \mathbf{F}^{T}) + (\partial_{\mathbf{F}} \mathbf{F})\mathbf{F}^{T}) \right) = 0\\ \mathrm{trace}\left( \mathbf{\Sigma}_D^{-1} \mathbf{F} (\partial_{\mathbf{F}} \mathbf{F}^{T})\mathbf{\Sigma}_D^{-1}\mathbf{S} + \mathbf{\Sigma}_D^{-1} (\partial_{\mathbf{F}} \mathbf{F}) \mathbf{F}^{T} \mathbf{\Sigma}_D^{-1}\mathbf{S} \right) - \mathrm{trace}\left( \mathbf{\Sigma}_D^{-1} \mathbf{F} (\partial_{\mathbf{F}} \mathbf{F}^{T}) + \mathbf{\Sigma}_D^{-1} (\partial_{\mathbf{F}} \mathbf{F})\mathbf{F}^{T} \right) = 0\\ $$

Using $\mathrm{trace}( \mathbf{A}+\mathbf{B}) = \mathrm{trace} (\mathbf{A})+\mathrm{trace}(\mathbf{B})$,

$$ \mathrm{trace}\left( \mathbf{\Sigma}_D^{-1} \mathbf{F} (\partial_{\mathbf{F}} \mathbf{F}^{T})\mathbf{\Sigma}_D^{-1}\mathbf{S}\right) + \mathrm{trace}\left(\mathbf{\Sigma}_D^{-1} (\partial_{\mathbf{F}} \mathbf{F}) \mathbf{F}^{T} \mathbf{\Sigma}_D^{-1}\mathbf{S} \right) - \mathrm{trace}\left( \mathbf{\Sigma}_D^{-1} \mathbf{F} (\partial_{\mathbf{F}} \mathbf{F}^{T})\right) - \mathrm{trace}\left(\mathbf{\Sigma}_D^{-1} (\partial_{\mathbf{F}} \mathbf{F})\mathbf{F}^{T} \right) = 0\\ $$

reparameterize matrices without effect to derivatives:

$$ \mathbf{A} \equiv \mathbf{\Sigma}_D^{-1} \mathbf{F}, \mathbf{B} \equiv \mathbf{\Sigma}_D^{-1}\mathbf{S}, \mathbf{C} \equiv \mathbf{F}^{T} \mathbf{\Sigma}_D^{-1}\mathbf{S}, \mathbf{D} \equiv \mathbf{F}^{T} $$

$$ \mathrm{trace}\left( \mathbf{A} (\partial_{\mathbf{F}} \mathbf{F}^{T})\mathbf{B}\right) + \mathrm{trace}\left( \mathbf{\Sigma}_D^{-1} (\partial_{\mathbf{F}} \mathbf{F}) \mathbf{C} \right) - \mathrm{trace}\left( \mathbf{A} (\partial_{\mathbf{F}} \mathbf{F}^{T})\right) - \mathrm{trace}\left(\mathbf{\Sigma}_D^{-1} (\partial_{\mathbf{F}} \mathbf{F})\mathbf{D} \right) = 0\\ $$

Using $\partial \mathrm{trace}(\mathbf{X}) = \mathrm{trace}(\partial \mathbf{X})$, transpose a derivative operator outside trace.

$$ \partial_{\mathbf{F}}\mathrm{trace}\left( \mathbf{A} \mathbf{F}^{T}\mathbf{B}\right) + \partial_{\mathbf{F}}\mathrm{trace}\left( \mathbf{\Sigma}_D^{-1} \mathbf{F} \mathbf{C} \right) - \partial_{\mathbf{F}}\mathrm{trace}\left( \mathbf{A} \mathbf{F}^{T}\right) - \partial_{\mathbf{F}}\mathrm{trace}\left(\mathbf{\Sigma}_D^{-1} \mathbf{F}\mathbf{D} \right) = \mathbf{0}\\ $$

Using the following derivatives of trace:

$$ \partial_{\mathbf{X}}\mathrm{trace}\left( \mathbf{A} \mathbf{X}^{T}\mathbf{B}\right)=\mathbf{B}\mathbf{A} , \partial_{\mathbf{X}}\mathrm{trace}\left( \mathbf{A} \mathbf{X}\mathbf{B}\right)=\mathbf{A}^{T}\mathbf{B}^{T} , \partial_{\mathbf{X}}\mathrm{trace}\left( \mathbf{X}\mathbf{A} \right)=\mathbf{A}^{T} , \partial_{\mathbf{X}}\mathrm{trace}\left( \mathbf{A}\mathbf{X}^{T} \right)=\mathbf{A} $$

So,

\begin{eqnarray} \mathbf{B}\mathbf{A} + (\mathbf{\Sigma}_D^{-1})^{T} \mathbf{C}^{T} - \mathbf{A} - (\mathbf{\Sigma}_D^{-1})^{T} \mathbf{D}^{T} &=& \mathbf{0} \\ \mathbf{\Sigma}_D^{-1}\mathbf{S}\mathbf{\Sigma}_D^{-1} \mathbf{F} + \mathbf{\Sigma}_D^{-1} (\mathbf{F}^{T} \mathbf{\Sigma}_D^{-1}\mathbf{S})^{T} - \mathbf{\Sigma}_D^{-1} \mathbf{F} - \mathbf{\Sigma}_D^{-1}(\mathbf{F}^{T})^{T} &=& \mathbf{0}\\ \mathbf{\Sigma}_D^{-1}\mathbf{S}\mathbf{\Sigma}_D^{-1} \mathbf{F} + \mathbf{\Sigma}_D^{-1} \mathbf{S}\mathbf{\Sigma}_D^{-1}\mathbf{F} - \mathbf{\Sigma}_D^{-1} \mathbf{F} - \mathbf{\Sigma}_D^{-1} \mathbf{F} &=& \mathbf{0}\\ 2\mathbf{\Sigma}_D^{-1} \mathbf{F} &=& 2\mathbf{\Sigma}_D^{-1}\mathbf{S}\mathbf{\Sigma}_D^{-1} \mathbf{F}\\ \mathbf{\Sigma}_D^{-1} \mathbf{F} &=& \mathbf{\Sigma}_D^{-1}\mathbf{S}\mathbf{\Sigma}_D^{-1} \mathbf{F}\\ \end{eqnarray}

it can derive eq.21.2.3 .

$\endgroup$
  • $\begingroup$ Kudos for answering your question and sharing it! (+1) $\endgroup$ – usεr11852 Jul 30 at 19:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.