how does `subsample` parameter work in boosting algorithms like xgboost and lightgbm? From what I know, both of them are sequential learners and only the 1st tree in the sequence gets built on the data and all the following trees that get built are to correct the mistakes of previous tree, hence improving the performance or decreasing the bias.
subsample parameters in xgboost and lightgbm dictates the percentage of rows used per tree building.
So, with this context, if subsample is set to 0.75, first tree gets built with 75% of the data and all the following trees will focus on correcting mistakes. So, what happens to the remaining 25% of the data? will another set of sequential tress be built parallelly? or am I missing something here or got something wrong?
 A: That 25% of the data is unused by that learner (i.e. that iteration) of the XGBoost model (assuming subsample=0.75). This is normal, what they do via the subsample argument is to implement bagging by subsampling once in every boosting iteration. This means that, as you described, a portion of the data is not used by that specific base-learner during the $i$-th iteration. In the $i+1$-th iteration, sub-sampling of the whole dataset is performed once again.
Through bagging (or more formally bootstrap aggregation) we practically bootstrap our estimator and allow ourselves to have a more robust overall result - think of it as estimating the sample mean via bootstrapping; just the "mean" here is the "expected prediction for each item" in our sample instead of a single "expected prediction for the sample's central tendency".
And note that irrespective of the subsampling proportion we might use (e.g. 10%), we can always provide estimates for all (i.e. 100%) of our sample items. Some will be out-of-sample of course and some in-sample. In that way, we can also estimate error gradients for all our sample items if needed.
A: I found other answers are confusing. Here is my understanding. XGBoost trees estimate the residual from the previous tree (we set a base_score as the 0th tree residual). When subsample is defined (rather than default 1.0), each tree is trained using the subsampled data, and each tree computes the residuals of all data; this makes sure you have all residuals to subsample for consequent trees. n_estimators are the number of your trees; it is not a bad idea to think of it as the iteration number in DNN, because you can also early stop the training before all trees are trained using early_stopping_rounds hpyerparameter. Hope it helps.
