3
$\begingroup$

Let's say we have some data X and we want to find a linear separator using soft SVM with l2 regularization, and then we want to solve the same problem after applying some rotation matrix Q to the data set. Should we expect anything to change in the accuracy over the training set?

The way I think about it, it doesn't make sense for anything to change because rotation matrix preserves norm and the its transpose is also its inverse, so mathematically:

enter image description here

So the solution should be identical and solving the two problems should yield the same accuracy and the same loss (having the same loss function).

Edit: Forgot to mention that the loss function of soft SVM is convex, so should have only one minimum.

Is something about this reasoning wrong? Is there something I'm missing?

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.