Can someone explain the RBF Kernel to me? I have read every explanation out there on this but nobody seems capable of explaining this in a way that I am able to understand. For an SVM RBF Kernel we often say that:

But what does x and x' represent here? Let's say my input data x is of shape 5x100 (5 features aka dimensions aka columns) and 100 rows, and output y is 1x100 rows, then what is x and x' in relation to this?
 A: The kernel function is a measure of similarity between two sets of features. So in this case, $x'$ and $x$ will both be $5\times 1$ feature vectors (not necessarily the same). $K(x,x')$ is a scalar that represents the similarity between $x$ and $x'$, and the kernel matrix $[K(x,x')]_{x\in X, x'\in X}$ is a $100\times 100$ matrix which represents the pairwise similarities.
The kernel function can be thought of as a cheap way of computing an infinite dimensional inner product - this 'kernel trick' is described in more detail in these notes. This lets the algorithm learn arbitrarily complex functions (though it may take an infinite number of samples for it to learn). 
A: Here's my attempt at giving a non-mathy explanation for the RBF kernel. The kernel function gives you the distance between $x$ and $x'$. It's not like the regular Euclidean distance though. I imagine living in an RBF kernel world like walking around in a little bubble where anything what is within arms reach of me is just like normal Euclidean distances but anything farther than that (outside the bubble) is warped to be extremely far The way this works with the math is $\sigma$ is like the size of the bubble and the $\exp$ function is what pushes everything to be really far away if it is not already close.
