Can somebody tell me how Tipping (2001) in his classical paper about the Relevance Vector Machine arrives at the following expressions \begin{align} \mathbf{D} &= (\mathbf{C}^{-1}\mathbf{t}\mathbf{t}^{T}\mathbf{C}^{-1}-\mathbf{C}^{-1}) \mathbf{\Phi} \mathbf{A}^{-1} \\ &= \beta[(\mathbf{t}-\mathbf{y})\boldsymbol{\mu}^{T} - \mathbf{\Phi}\Sigma] \end{align}

where \begin{equation} \mathbf{C} = \beta^{-1}\mathbf{I} + \mathbf{\Phi} \mathbf{A}^{-1} \mathbf{\Phi}^{T}, \\ \boldsymbol{\mu} = \beta^{-1}\mathbf{\Sigma}\mathbf{\Phi}^{T}\mathbf{t}, \\ \boldsymbol{\Sigma} = \left( \mathbf{A} + \beta \mathbf{\Phi}^{T} \mathbf{\Phi} \right)^{-1} \end{equation}

which contain the derivatives of likelihood \begin{equation} \mathcal{L} = -\frac{1}{2}\big[N\log 2\pi + \log\left|\mathbf{C}\right| + \mathbf{t}^{T}\mathbf{C}^{-1}\mathbf{t} \big] \end{equation}

with respect to the elements $\phi_{mn}$ of the design matrix $\mathbf{\Phi}$ (i.e. $\mathbf{D}_{mn} = \partial \mathcal{L} / \partial \phi_{mn}$)?

Note that bold small letters e.g $\mathbf{t}$ denote $N\times 1$ vectors, bold capital letters $N\times N$ matrices, and $\mathbf{A}$ is diagonal. Also, the author makes frequent use of the following identity (by the Woodbury inversion lemma) which might be helpful:

\begin{align} \mathbf{C}^{-1} = (\beta^{-1}\mathbf{I} + \mathbf{\Phi} \mathbf{A}^{-1} \mathbf{\Phi}^{T} )^{-1} = \beta \mathbf{I} - \beta \mathbf{\Phi}\left( \mathbf{A} + \beta \mathbf{\Phi}^{T} \mathbf{\Phi} \right)^{-1} \mathbf{\Phi}^{T} \beta = \beta \mathbf{I} - \beta \mathbf{\Phi} \mathbf{\Sigma} \mathbf{\Phi}^{T} \beta \end{align}


1 Answer 1


I also asked on math.stackexchange and got an answer to the above question.

  • $\begingroup$ Why the downvote? Or to frame it differently, what else could I have done to lead people from this post to the answer? $\endgroup$
    – mwater
    Oct 22, 2018 at 19:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.