# Why training error > 0 on SVM with RBF kernel

When using RBF kernel I think feature space is infinite dimensional space. With infinite dimensional features, I believe any training set can be classified. So I'm wondering why training error > 0 even then? Does smoothing factor occurs that?

### update

I forgot to mention about the identical feature vectors and the opposite label. Please ignore that case.

• What Happens if two examples have identical feature vectors but opposite labels? – Sycorax May 14 '16 at 4:50
• What if you have 1000 irrelevant features and one moderately relevant feature? What if all features are irrelevant? – Sycorax May 14 '16 at 8:46
• @C11H17N2O2SNa if all features are irrelevant, we can still get 0 error by sort of "memorizing" the mapping. i'm not sure is that doable with RBF kernel? to overfit everything? – dontloo May 14 '16 at 10:13

the infinite feature space of rbf consists of a $exp(-x^2)$ term and infinite $\sqrt{\frac{2^k}{k!}}x^k$ terms with $k=0$ to infinity, according to the Taylor series for the exponential function.