Tree-based methods, e.g., random forests or GBRT use trees to construct most informative space partitioning. Given a single tree, in each partition, the value of the regression function (or a class) is computed using a simple model. For example, it can be a constant or a single label. Catboost's regression trees AFAIK can fit a linear regression.
In contrast, even simple 1-2 hidden layer neural networks (with non-linear activation) can beat GBRTs or random forests although it is not clear how a couple of matrix multiplications can apparently create region-specific regression or classification functions. Region-specific is a bit of a misnomer, but the point is that neural networks can apparently identify regions of interest and behave differently in different regions.
Is there a simple explanation of how this happens? Is there a complicated one (e.g., some theory)?