Why would a BaggingRegressor only use a subset of samples and features during fitting?

Question

I am working with a BaggingRegressor from sklearn and am having difficulty understanding what the purpose/effect of max_features and max_samples is on the model fit. From the description of the attributes

max_samples : int or float, optional (default=1.0)
    The number of samples to draw from X to train each base estimator.
            If int, then draw max_samples samples.
            If float, then draw max_samples * X.shape[0] samples.

max_features : int or float, optional (default=1.0)
    The number of features to draw from X to train each base estimator.
            If int, then draw max_features features.
            If float, then draw max_features * X.shape[1] features.

The ability to change these seems a bit odd to me, especially seeing the default values. I guess I can see why from a programming standpoint they might be set to 1 by default (handle small data sets), but from an actual use case standpoint these seem like they should almost never be used, is that a fair assumption?

---

Overall I do not see many reasons why these values should not be set to len(X.shape[0]) and len(X.shape[1]) respectively; using all samples and all features.

If you used less samples

• You could "tune" how the OOB score is calculated.
• ...?

If you use less predictors

• Try to handle cases where the number of predictors is not much larger than the number of observations. (Curse of dimensionality)
• Possibly simulate a random forest.

---

Why would a model ever want to set these attributes to anything different that their respective maximums?

How do these attributes effect the model fitting?

Answer 1

The purpose of these tweaks to bagging is to decrease the correlation between bagged regressors.

Why limit the allowed features? If you allowed complete freedom to each regressor, then one feature may come to dominate most of the fitted regressors. By purposefully leaving our features, you are adding some bias to the estimator with the hope that it's predictions will be less correlated with the other fitted regressors. By reducing the correlation, you get a larger variance reduction through bagging. As long as the added bias is small, then you get a boost in performance.

As for the number of samples, the same rule applies. You are trying to avoid your regressors fixating on a peculiarity of your dataset, so you purposefully thin it to provide more varied datasets to your regressors, at the expense of increasing the variance somewhat.

Again, it all comes down to decreasing correlation by adding bias and/or variabiilty to your base estimators. The decreased correlation makes bagging more effective as long as the added bias/variability is not too large.