I am trying to make a regression with SVR and I found a problem in the process, the regression with random data is ok, but I tried it with my data, and with all of these three kernels the prediction's output is constant (see the plot). Here is a piece of my data, maybe the problem is here, but I cant'see why.



And here is the code I'm using.

import pandas as pd
from sklearn.svm import SVR
import matplotlib.pyplot as plt
import numpy as np

#Importing data
data = pd.read_csv('data.csv')
data = data.as_matrix()

#Random data generator
#datar = np.random.random_sample((7,21))
#inputdatar = datar[:,0:4]

inputdata = data[:,0:4]
output1 = data[:,4]

svr_rbf = SVR(kernel='rbf',gamma=1)
pre = svr_rbf.predict(inputdata)
axis = range(0,data.shape[0])

plt.scatter(axis, output1, color='black', label='Data')
plt.plot(axis, pre, color='red', label='Regression')

enter image description here

I think maybe it's hyperparameter tuning problem, but I'm not sure if the data would cause a problem as well. Any lights?

  • $\begingroup$ Where's the plot? And what is your polynomial implementation? $\endgroup$
    – Archie
    Dec 30, 2016 at 12:50
  • $\begingroup$ I'll add the plot. The polynomial implementation i was refering to change SVR's kernel from rbf to poly. $\endgroup$ Dec 30, 2016 at 12:52
  • $\begingroup$ I edited the question a little bit, think doesn't change the answer it all. $\endgroup$ Dec 30, 2016 at 12:59

2 Answers 2


I believe the problem is that your data (in particular, the target variable) isn't scaled.

The SVR implementation in scikit-learn has a parameter, epsilon, that controls the loss function. Quoting from the docs, "It specifies the epsilon-tube within which no penalty is associated in the training loss function with points predicted within a distance epsilon from the actual value."

The default value of epsilon is 0.1. As you can see, all of your predictions are within 0.1 of the true value, so the loss is zero.

I'd fix this by scaling (normalizing) your data, or by using a different (smaller) value of epsilon. For instance, with epsilon equal to 0.001, I get a very non-linear curve that fits the data perfectly (probably not what you want either, to be fair).

  • $\begingroup$ Thanks, @vbox! Do you think normalizing is better than smaller epsilon? $\endgroup$ Dec 31, 2016 at 16:43
  • $\begingroup$ In general, I'd say that normalizing is a better approach. For instance, SVR uses regularization, and it is well-known that regularization only makes sense when the features have been normalized first. See for instance, this post. $\endgroup$
    – vbox
    Dec 31, 2016 at 17:14

That's a matter of scaling your data. I believe you can solve that simply by using a StandardScaler from sklearn Here is some sample code that I used:

# For SVR we need Feature Scaling
from sklearn.preprocessing import StandardScaler
sc_x = StandardScaler()
sc_y = StandardScaler()
# Scale x and y (two scale objects)
x = sc_x.fit_transform(x)
y = sc_y.fit_transform(y)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.