sklearn.svm.LinearSVR implementation, the default parameter of
epsilon is 0.0. And the documentation says "if unsure, set this to 0" via the Scikit LinearSVR doc. I am not trained in statistics but it's my understanding that the power of the LinearSVR techniques comes from the fact that it can solve an optimization problem that it disregards makes errors within an
epsilon distance. If that's the case, is there any value to train a LinearSVR model with an epsilon as 0?
It does make sense.
Setting $\epsilon \leftarrow 0$ makes the SVM equivalent to Mean Absolute Deviation regression, plus the $\ell_2$ penalization.
Optimizing the mean-squared error corresponds to optimizing for a Gaussian likelihood, while optimizing for mean absolute deviation corresponds to optimizing for a Laplace distribution.