addition of a kernel and a function I have this problem. 

My approach would be to take a Kernel matrix of $ \ k(x,y) $ as it is positive semidefinitive. And if I am able to make $ \ k'(x,y) $ a kernel matrix, then I can prove that it is a kernel. But I don't understand how can I use this in the proof $ \ δ(x, y) $
 A: Let $x \in \mathbb R^n$ and let $K_{ij} = k(x_i, x_j)$, $\Delta_{ij} = \delta(x_i, x_j)$. Then for any non-zero $y \in \mathbb R^n$ we have
$$
y^T (K + \Delta)y = \underbrace{y^T K y}_{\geq 0} + y^T\Delta y
$$
so we need to look at $y^T \Delta y \stackrel ?{\geq} 0$ in order to decide if $k'$ is PSD.
For $y^T \Delta y$ we have
$$
y^T \Delta y = \sum_{ij} y_iy_j\delta_{ij} = \sum_i y_i^2 \delta_{ii} = y^T y > 0
$$
since $y\neq 0$. We could have also just noted that $\Delta = I$ so $y^T \Delta y = y^T I y = y^T y > 0$.
This means that $K + \Delta$ is PD, not even just PSD, and so $k'$ is too. Intuitively, PD matrices are ones that are diagonally dominant and we've just added $1$ to the diagonal of $K$. We can also think of this as adding $1$ to each eigenvalue of $K$, which again guarantees invertibility as it is already PSD so now all eigenvalues are bounded away from $0$.
Finally, if $K = X^TX$ (i.e. we're using the linear kernel) then $K + \Delta$ is exactly a ridge regression with $\lambda = 1$. We just proved a special case of the result that $X^TX + \lambda I$ is always invertible for $\lambda > 0$.
A: @Chaconne proof is better than mine as it give more details. But if you want to use results about kernels to avoid all the calculations, you can use the following results.
The limit of a sequence of kernels is a kernel
If $\kappa_1, \kappa_2, \dots$ are kernels, and $\kappa(x, y) := \lim_{n \to \infty} \kappa_n(x, y)$ exists for all $x, y$, then $\kappa$ is a kernel.
The sum of two kernels is a kernel
If $\kappa_1$ and $\kappa_2$ are kernels, so is $\kappa_1 + \kappa_2$.
Now, note that $\delta = \lim_{\sigma\rightarrow \infty} (x,y\rightarrow \exp(-\frac{||x-y||^2)}{2\sigma^2}))$, the pointwise limit of the gaussian kernel. So $\delta+\kappa$ is a kernel, as a sum of two kernels.
