# How do I prove the following equality?

Let $\Phi = \begin{bmatrix} 1 & \varphi_1(x_1) & \varphi_2(x_1) & \dots & \varphi_{M-1}(x_1) \\ 1 & \varphi_1(x_2) & \varphi_2(x_2) & \dots & \varphi_{M-1}(x_2) \\ \vdots \\ 1 & \varphi_1(x_N) & \varphi_2(x_N) & \dots & \varphi_{M-1}(x_N) \end{bmatrix}=\begin{bmatrix}\phi(x_1)^T\\ \vdots\\ \phi(x_N)^T\end{bmatrix}$

where $\varphi_m(x)$ is the $m^{th}$ basis function for regression

$y = 1 + \sum_{m=1}^{M-1}w_m\varphi_m(x)$

How can I prove that

$\sum_{n=1}^Nk(x,x^n)=\sum_{n=1}^N\phi(x)^T(\Phi^T\Phi)^{-1}\phi(x_n)=1$

where $k$ is the equivalent kernel for Bayesian linear regression. $w_m$ are the regression coefficients whose MLE solutions are already included in the equivalent kernel.

[Update]

Okay. Looks like I have done a lousy job describing my question. My apologies. My question is really simple and it does not need more context than what I have already described. All I want to prove is given a simple design matrix of say for example a ${M-1}^{th}$ order polynomial regression $\Phi=$

$\begin{bmatrix} 1 & x_1 & x_1^2 & \dots & x_1^{M-1} \\ 1 & x_2 & x_2^2 & \dots & x_2^{M-1} \\ \vdots \\ 1 & x_N & x_N^2 & \dots & x_N^{M-1} \end{bmatrix}$

which can be written in a more compact form

$\Phi=\begin{bmatrix}\phi(x_1)^T\\ \vdots\\ \phi(x_N)^T\end{bmatrix}$

where $\phi(x_n)^T$ is the $n^{th}$ row of $\Phi$, $\phi(x_n)^T=[1,x_n,...,x_n^{M-1}]$. So each row of $\Phi$ is a mapping of an input x to an M-dimensional feature space which is spanned by the columns of $\Phi$. Also given a new $x^*$ with its corresponding mapping to the M-dimensional space, $\phi(x^*)^T=[1,x^*,...,{x^*}^{M-1}]$,prove that the following matrix operation yields a scalar 1.

$\sum_{n=1}^N\phi(x^*)^T(\Phi^T\Phi)^{-1}\phi(x_n)=1$

• If it is a probability density, that might be an axiom. – EngrStudent - Reinstate Monica Nov 5 '17 at 1:09
• You need to give much more context than that. – Yair Daon Nov 5 '17 at 2:23
• @YairDaon Sorry about that. I have added more detail – Karlsson Yu Nov 5 '17 at 5:10
• What do you mean by "$k$"? What are the "$\phi_n$"? – whuber Nov 5 '17 at 17:33
• @whuber $k$ is the equivalent kernel, $\phi(x)$ is a column vector of the basis functions – Karlsson Yu Nov 5 '17 at 23:36

## 1 Answer

Never mind, I think I proved it. Btw, the proof for Eq.3.64 provided by Bishop in the Machine learning and pattern recognition might not be correct.

$\sum_{n=1}^Nk(x,x^n)=\sum_{n=1}^N\phi(x)^T(\Phi^T\Phi)^{-1}\phi(x_n)$ can be rewritten as

$\sum_{n=1}^N\phi(x)^T(\Phi^T\Phi)^{-1}\phi(x_n)\times 1=\phi(x)^T(\Phi^T\Phi)^{-1}\Phi^T1_N$

let $\Phi^T1_N = a_1$. In fact $a_1$ is the first column of $A=\Phi^T\Phi$

Using the fact that $A^{-1}A=I=[e_1,...e_M]$, we have $A^{-1}a_1=e_1$

Therefore, $\sum_{n=1}^Nk(x,x_n)=\phi(x)^T(\Phi^T\Phi)^{-1}\Phi^T1_N=\phi(x)^Te_1=1$, according to the definition of $\Phi$, $\phi(x)^T=[1,\varphi_1(x),...,\varphi_{M-1}(x)]$

This proof only holds when there is always a bias term of 1 added to the Bayesian linear regression function

• I think your proof might be wrong in several ways but it is so hard to tell with (still) so little context. – Yair Daon Nov 6 '17 at 2:57
• This is all getting a bit of a mess. Why not start again with a new question with all the context and link it back to here? – mdewey Nov 6 '17 at 9:58
• @YairDaon I have added more explanation of the problem – Karlsson Yu Nov 6 '17 at 11:17