I was going over https://en.wikipedia.org/wiki/Cover%27s_theorem And I am a bit lost with the intuition.
I do understand that if it's not linearly separable, then projecting it into a higher-dimensional space shall solve it (it can be nicely explain with a real piece of paper).
The "projecting it into a higher-dimensional space" makes me uncomfortable since its basically introducing a new column into the dataset with unrelated values...
I feel it's cheating, the data was in a lower-dimensional space for a reason... You may now be able to separate data but its not the original data, some arbitrary new "column" has been added.
Isn't it better trying to find a non-linear separation, instead of projecting into a higher dimension?