# regression with scikit-learn with multiple outputs, svr or gbm possible?

I have been trying regression with scikit-learn with a problem with multiple outputs like this:

X = np.random.random((10,3))
y = np.random.random((10,2))
X2 = np.random.random((7,3))
clf = SVR()
clf.fit(X, y)
y_pred = clf.predict(X2)


The problem is that this doesn't work. It fails with:

ValueError: Buffer has wrong number of dimensions (expected 1, got 2)


Does anyone know how to deal with regression with multiple outputs in scikit-learn?

Edit. I have noticed RandomForestRegressor, KNeighborsRegressor, and LinearRegression all work, but rf is the only one that's close to being good on my dataset! Is there some way to fit SVR or gbm?

Why not make a wrapper that would fit m regressors (where m is dimensionality of each y) like this?

class VectorRegression(sklearn.base.BaseEstimator):
def __init__(self, estimator):
self.estimator = estimator

def fit(self, X, y):
n, m = y.shape
# Fit a separate regressor for each column of y
self.estimators_ = [sklearn.base.clone(self.estimator).fit(X, y[:, i])
for i in range(m)]
return self

def predict(self, X):
# Join regressors' predictions
res = [est.predict(X)[:, np.newaxis] for est in self.estimators_]
return np.hstack(res)


Note: I haven't tested this code, but you got the idea.

• Thanks, i was thinking about something like that.. The only downfall is the problem has 111 outputs! At 1 minute each that would take almost 2 hours! I suppose the list comprehensions could be parallelized here though.. – anthonybell May 26 '15 at 3:53
• The sklearn MultiOutputRegressor is just such a class – Zach Jul 11 '19 at 18:42

Scikit-Learn also has a general class, MultiOutputRegressor, which can be used to use a single-output regression model and fit one regressor separately to each target.

Your code would then look something like this (using k-NN as example):

from sklearn.neighbors import KNeighborsRegressor
from sklearn.multioutput import MultiOutputRegressor

X = np.random.random((10,3))
y = np.random.random((10,2))
X2 = np.random.random((7,3))

knn = KNeighborsRegressor()
regr = MultiOutputRegressor(knn)

regr.fit(X,y)
regr.predict(X2)


I think that scikit-learn only supports multi-output regressors in decision tress: DecisionTreeRegressor.

• Yes, I have noticed that only RandomForestRegressor and LinearRegression seem to work out of the box for multiple output regression. There isn't some class to fit single output regressors into multiple output regressors? Where you can put in any regressor and it will internally call it for each output? – anthonybell May 25 '15 at 2:06