I am wondering, for the Gaussian kernel
$$k(x_n,x_m) = \exp(- \frac{\theta}{2} \lVert x_n-x_m\rVert^2)$$
whether we really need the exponent.
What are the consequences of just using
$$k(x_n,x_m) = \frac{\theta}{2} \lVert x_n-x_m\rVert^2$$
I am wondering, for the Gaussian kernel
$$k(x_n,x_m) = \exp(- \frac{\theta}{2} \lVert x_n-x_m\rVert^2)$$
whether we really need the exponent.
What are the consequences of just using
$$k(x_n,x_m) = \frac{\theta}{2} \lVert x_n-x_m\rVert^2$$
One pretty big consequence is that it's no longer a positive semidefinite kernel, so that the mathematical assumptions of almost all kernel methods don't apply. (Relevant discussion focused on Gaussian Processes here.)
Your proposed kernel has $k(x, x) = 0$: all points have zero "self-similarity." Thus for any $x_1 \ne x_2$, the kernel matrix given by the two points will be of the form $\begin{bmatrix}0 & c \\ c & 0\end{bmatrix}$ with $c > 0$, which is not positive semidefinite.
Another vaguely similar and actually useful kernel is the "distance kernel": $$ k(x, y) = \lVert x - O \rVert + \lVert y - O \rVert - \lVert x - y \rVert ,$$ where $O$ is any fixed point in space, e.g. the origin. This is implicitly the kernel used by the famous "energy distance" / "distance covariance" when viewed as a kernel MMD (paper).