Does, in some sense, Kernel Ridge Regression refer to the same class of problems as Support Vector Regression? Should I use them almost equivalently?
Yes, they refer to the same class of problems, aka. fitting a linear function to data in possibly transformed space, but they don't solve the same optimization problem (they have different loss functions), and will give different results.
To see the exact difference you may want to see chapters $3.4.1$ and $12.3.6$ of Elements of Statistical learning II, the e-book can be found for free here