1
$\begingroup$

Can somebody tell me how Tipping (2001) in his classical paper about the Relevance Vector Machine arrives at the following expressions \begin{align} \mathbf{D} &= (\mathbf{C}^{-1}\mathbf{t}\mathbf{t}^{T}\mathbf{C}^{-1}-\mathbf{C}^{-1}) \mathbf{\Phi} \mathbf{A}^{-1} \\ &= \beta[(\mathbf{t}-\mathbf{y})\boldsymbol{\mu}^{T} - \mathbf{\Phi}\Sigma] \end{align}

where \begin{equation} \mathbf{C} = \beta^{-1}\mathbf{I} + \mathbf{\Phi} \mathbf{A}^{-1} \mathbf{\Phi}^{T}, \\ \boldsymbol{\mu} = \beta^{-1}\mathbf{\Sigma}\mathbf{\Phi}^{T}\mathbf{t}, \\ \boldsymbol{\Sigma} = \left( \mathbf{A} + \beta \mathbf{\Phi}^{T} \mathbf{\Phi} \right)^{-1} \end{equation}

which contain the derivatives of likelihood \begin{equation} \mathcal{L} = -\frac{1}{2}\big[N\log 2\pi + \log\left|\mathbf{C}\right| + \mathbf{t}^{T}\mathbf{C}^{-1}\mathbf{t} \big] \end{equation}

with respect to the elements $\phi_{mn}$ of the design matrix $\mathbf{\Phi}$ (i.e. $\mathbf{D}_{mn} = \partial \mathcal{L} / \partial \phi_{mn}$)?

Note that bold small letters e.g $\mathbf{t}$ denote $N\times 1$ vectors, bold capital letters $N\times N$ matrices, and $\mathbf{A}$ is diagonal. Also, the author makes frequent use of the following identity (by the Woodbury inversion lemma) which might be helpful:

\begin{align} \mathbf{C}^{-1} = (\beta^{-1}\mathbf{I} + \mathbf{\Phi} \mathbf{A}^{-1} \mathbf{\Phi}^{T} )^{-1} = \beta \mathbf{I} - \beta \mathbf{\Phi}\left( \mathbf{A} + \beta \mathbf{\Phi}^{T} \mathbf{\Phi} \right)^{-1} \mathbf{\Phi}^{T} \beta = \beta \mathbf{I} - \beta \mathbf{\Phi} \mathbf{\Sigma} \mathbf{\Phi}^{T} \beta \end{align}

$\endgroup$
-1
$\begingroup$

I also asked on math.stackexchange and got an answer to the above question.

$\endgroup$
  • $\begingroup$ Why the downvote? Or to frame it differently, what else could I have done to lead people from this post to the answer? $\endgroup$ – mwater Oct 22 '18 at 19:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.