Why are these proper kernels and how to deduce that they are? I am struggling to understand kernels and how to determine whether they are proper or not.
For these examples can anyone explain why an example is proper and why another example is not.
Given $K_1$ and $K_2$ two proper kernels, determine which of the following formulae define proper kernels:
$$
K_4(x_1,x_2) = -K_1(x_1,x_2)^2 K_2(x_1,x_2)
\\
K_3(x_1,x_2) = -K_2(x_1,x_2) + 3K_1(x_1,x_2)
\\
K_5(x_1,x_2) = 15 K_1(x_1,x_2)
$$
I am particularly interested in how you get the answer, rather than stating which are proper and which are not.
 A: In a machine learning context (i.e. "kernel methods"), the key requirement for a kernel is that it must be symmetric and positive-definite, that is, if $K$ is a kernel matrix, then for any (column) vector $x$ of the appropriate length, $x^{T}Kx$ must be a positive real number. This restriction is in place mostly due to requirements of optimization processes that operate downstream on this matrix.
To answer your question, certain basic operations preserve positive-definiteness, and some don't. You can use the definition of positive-definiteness above to decide whether an operation preserves that property or not. The product operation does not always preserve positive definiteness (it does preserve it in the case where their product is commutative). The square operation however does preserve it. For a real positive number $r$, $rK$ is positive definite, and the sum of any two positive definite matrices is also positive definite.
So in your examples, $K_3$ and $K_4$ may or may not be proper kernel matrices (to use your terminology) and $K_5$ is definitely a proper kernel matrix.
A: One easy way is to think of a kernel function as a positive-semidefinite (PSD) matrix. Then you can use PSD tricks.
For example, if $K$ is PSD, then $-K$ is NSD and thus not PSD. On the other hand, if $K$ is PSD, $c  K$ is PSD if $c$ is a scalar and $c > 0$. Thus, $K_4$ and $K_3$ are not generally kernels and $K_5$ is.
