Let's say we have some data X and we want to find a linear separator using soft SVM with l2 regularization, and then we want to solve the same problem after applying some rotation matrix Q to the data set. Should we expect anything to change in the accuracy over the training set?

The way I think about it, it doesn't make sense for anything to change because rotation matrix preserves norm and the its transpose is also its inverse, so mathematically:

enter image description here

So the solution should be identical and solving the two problems should yield the same accuracy and the same loss (having the same loss function).

Edit: Forgot to mention that the loss function of soft SVM is convex, so should have only one minimum.

Is something about this reasoning wrong? Is there something I'm missing?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.