Proof of sum of kernels of concatenated vector I am reading the Pattern Recognition and Machine Learning book by Bishops.
I have some difficulties proving the (6.21) equation. Can someone help me?

N.B: If someone has a better title for this question, i'm opened to it as i couldn't find the good words.
 A: Here are two proofs:
Gram matrix is psd:
For any set of points $\{x_i\}_{i=1}^n$, call the Gram matrix for $k_a$ $A$ and the Gram matrix for $k_b$ $B$. Then the Gram matrix for $k$ is just $A + B$, and we have
$$
\alpha^T (A + B) \alpha = \alpha^T A \alpha + \alpha^T B \alpha \ge 0
$$
since $\alpha^T A \alpha \ge 0$, $\alpha^T B \alpha \ge 0$.
Thus $k$ is a psd kernel.
Feature maps:
This is effectively taking the direct sum of two kernels.
Let $\mathcal H_a$ be the RKHS for kernel $k_a$, and $\mathcal H_b$ for $k_b$, with feature maps $\varphi_a : \mathcal X_a \to \mathcal H_a$ and $\varphi_b : \mathcal X_b \to \mathcal H_b$. Define the RKHS $\mathcal H = \mathcal H_a \oplus \mathcal H_b$, where an arbitrary element of $\mathcal H$ is $f_a \oplus f_b$.
(If $\mathcal H_a = \mathbb R^{d_a}$ and $\mathcal H_b = \mathbb R^{d_b}$ are Euclidean spaces, then $\mathcal H = \mathbb R^{d_a + d_b}$ and $f_a \oplus f_b$ is just the concatenation of $f_a$ and $f_b$.)
The inner product is then
$$\langle f_a \oplus f_b, f_a' \oplus f_b' \rangle_{\mathcal H} = \langle f_a, f_a' \rangle_{\mathcal H_a} + \langle f_b, f_b' \rangle_{\mathcal H_b}.$$
So we just need to define $\varphi(x) = \varphi_a(x_a) \oplus \varphi_b(x_b)$, in which case we get
\begin{align}
k( x, x' )
&= \langle \varphi(x), \varphi(x') \rangle_{\mathcal H}
\\&= \langle \varphi_a(x_a) \oplus \varphi_b(x_b), \varphi_a(x_a') \oplus \varphi_b(x_b') \rangle_{\mathcal H}
\\&= \langle \varphi_a(x_a), \varphi_a(x_a') \rangle_{\mathcal H_a}
   + \langle \varphi_b(x_b), \varphi_b(x_b') \rangle_{\mathcal H_b}
\\&= k_a(x_a, x_a') + k_b(x_b, x_b')
.\end{align}
We can easily verify this a valid RKHS (sum of two continuous functionals is continuous), and so $k$ is again a psd kernel.
