Why SVR kernels other than 'linear' don't work for this toy dataset! I'm a little new with modeling techniques and I'm trying to compare SVR and Linear Regression. I've used f(x) = 5x+10 linear function to generate training and test data set. Here we've discussed why SVR with rbf Kernel fails in prediction of such a simple dataset. This is the python code snippet with SVR linear kernel that we've tried. 
import numpy as np
import pandas as pd
from sklearn.linear_model import LinearRegression
from sklearn.svm import SVR
from sklearn.cross_validation import train_test_split

X = np.linspace(0,100,101)
y = np.array([(100*np.random.rand(1)+num) for num in (5*x+10)])

X_train, X_test, y_train, y_test = train_test_split(X, y)

svr = SVR(kernel='linear')
lm = LinearRegression()
svr.fit(X_train.reshape(-1,1),y_train.flatten())
lm.fit(X_train.reshape(-1,1), y_train.flatten())

pred_SVR = svr.predict(X_test.reshape(-1,1))
pred_lm = lm.predict(X_test.reshape(-1,1))

plt.plot(X,y, label='True data')
plt.plot(X_test[::2], pred_SVR[::2], 'co', label='SVR')
plt.plot(X_test[1::2], pred_lm[1::2], 'mo', label='Linear Reg')
plt.legend(loc='upper left');
plt.show()


Later by changing SVR kernel to rbf it fails to provide an acceptable solution. 

I'm really confused why this happens in case of such a toy dataset!?
 A: It seems as if this is a Bug/numerical instability in the scikit package (or we/you are using it wrongly although I can't see an obvious mistake...):
As far as I have understood from the documentation, scikit uses libsvm to do the actual optimization magic. Using R and its binding to libSVM (i.e. both approaches should compute exactly the same thing) I get senseful results:
R code:
install.packages("e1071")
library("e1071")
x = 1:100
y = 5*x + 10 + 100*runif(n = length(x), min = 0, max = 1)

binomial = sample.int(2, size=100, replace=T)-1
training_x = x[binomial==1]
training_y = y[binomial==1]

test_x = x[binomial==0]
test_y = y[binomial==0]

model = svm(training_x,training_y,kernel="radial",type = "eps-regression")
new = predict(model, test_x)

plot(x, y)
points(test_x, new, col = "red")

result: (red points are the ones predicted by SVM):

I would recommend to write this question to the owners of scikit-learn (including the R code in order to be able to compare).
Regards,
FW
A: The SVR call with the rbf kernel should be done while specifying the C and gamma options:
svr = SVR(kernel='rbf', C=1e3, gamma=0.1)

Below is the result using the rbf kernel:

