I have been trying regression with scikit-learn with a problem with multiple outputs like this:

X = np.random.random((10,3))
y = np.random.random((10,2))
X2 = np.random.random((7,3))
clf = SVR()
clf.fit(X, y)
y_pred = clf.predict(X2)

The problem is that this doesn't work. It fails with:

ValueError: Buffer has wrong number of dimensions (expected 1, got 2)

Does anyone know how to deal with regression with multiple outputs in scikit-learn?

Edit. I have noticed RandomForestRegressor, KNeighborsRegressor, and LinearRegression all work, but rf is the only one that's close to being good on my dataset! Is there some way to fit SVR or gbm?


4 Answers 4


Why not make a wrapper that would fit m regressors (where m is dimensionality of each y) like this?

class VectorRegression(sklearn.base.BaseEstimator):
    def __init__(self, estimator):
        self.estimator = estimator

    def fit(self, X, y):
        n, m = y.shape
        # Fit a separate regressor for each column of y
        self.estimators_ = [sklearn.base.clone(self.estimator).fit(X, y[:, i])
                               for i in range(m)]
        return self

    def predict(self, X):
        # Join regressors' predictions
        res = [est.predict(X)[:, np.newaxis] for est in self.estimators_]
        return np.hstack(res)

Note: I haven't tested this code, but you got the idea.

  • 1
    $\begingroup$ Thanks, i was thinking about something like that.. The only downfall is the problem has 111 outputs! At 1 minute each that would take almost 2 hours! I suppose the list comprehensions could be parallelized here though.. $\endgroup$ May 26, 2015 at 3:53
  • 4
    $\begingroup$ The sklearn MultiOutputRegressor is just such a class $\endgroup$
    – Zach
    Jul 11, 2019 at 18:42

Scikit-Learn also has a general class, MultiOutputRegressor, which can be used to use a single-output regression model and fit one regressor separately to each target.

Your code would then look something like this (using k-NN as example):

from sklearn.neighbors import KNeighborsRegressor
from sklearn.multioutput import MultiOutputRegressor

X = np.random.random((10,3))
y = np.random.random((10,2))
X2 = np.random.random((7,3))

knn = KNeighborsRegressor()
regr = MultiOutputRegressor(knn)


To answer the question from the edit. I guess that algorithms that naturally support MultiOutput targets, perform best. This is because these algorithms calculate the multiple output variables simultaneously and hence take possible correlations between outputs into account. This is not the case, if you use MultiOutputRegressor from sklearn which fits a model for each output variable individually.

SVR naturally only supports single-output regression. But there are different adaptions that can be made to make the algorithm fit also to a multi-output regression task. For an extensive overview check the paper in the Reference section of this repository.

You can find an example for an implementation of Multiple-output support vector regression in python here. It is based on the paper Multi-step-ahead time series prediction using multiple-output support vector regression.

You also might want to check out this answer.


I think that scikit-learn only supports multi-output regressors in decision tress: DecisionTreeRegressor.

  • $\begingroup$ Yes, I have noticed that only RandomForestRegressor and LinearRegression seem to work out of the box for multiple output regression. There isn't some class to fit single output regressors into multiple output regressors? Where you can put in any regressor and it will internally call it for each output? $\endgroup$ May 25, 2015 at 2:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.