No. We should focus our attention elsewhere, in particular to the validation approach we use.
In a situation that SNR is low it is especially pertinent to focus our attention to the validation schema in place. It is easy to be lulled in a false sense of security because we unintentionally over-fit our data or (less often) under-fit our data because we have regularised too aggressively without realising it.
It is good practice to use a repeated cross-validation schema (e.g. simple repeated CV) as well as to have a hold-out sample to ensure that our error metrics (e.g. RMSE and MAE) are comparable. If the two metrics are not generally comparable it might suggest that our algorithm potentially has poor generalisation performance despite our extensive resampling.
Similarly, it is important to anticipate potential clusters in our data (e.g. spatial or time-associations) that might make a simple resampling technique insufficient in terms of giving us "independent" sub-sets. An example from a past project I worked: we were performing a learning task where the unit of analysis was a residential flat; nevertheless, flats within the same building (i.e. block of flats) behaved so similarly that while our unit of analysis as a flat indeed, we were inadvertently leaking information. As soon as, we used building as the unit of resampling, the ability of our learner to generalise as well as its interpretability went up despite our CV metrics going nominally down.
Touching on the "under-fitting" too: As we increase the number of hyperparameters to optimise against we increase our search space. That means that our grid- or random-search needs to explore a wide search space so we have an adequately performant combination. The difficulty of optimization increases roughly exponentially with regard to the number of parameters; i.e. we need to exponentially increase the number of necessary trials when we increase the number of parameters. If two hyper-parameters have "roughly the same" influence then optimising both may not be the best use of our time. (Yes, their are some rule-of-thumbs that can be used (e.g. 60 trials in RS) but they not silver bullets nor give us strong guarantees that we haven't under-fitted.)
Please note that not all hyper-parameters matter equally. There is some work on this matter (e.g. see van Rijn & Hutter (2018) Hyperparameter Importance Across Datasets and references within). If one is willing to do a reasonable large exploration it might be worth looking at such work in more detail.
To conclude: Invest on the validation schema. Given we have a reasonable CV schema in place optimising for many additional hyper-parameters starts having diminishing returns. In general, I would always optimise for max_depth
because it has the most profound impact on the quality of fit and the complexity of the learner before any of the other three hyperparameters mentioned. That said, given you use min_sum_hessian_in_leaf
and min_gain_to_split
I think you won't gain a lot more by optimising for max_depth
too (assuming that max_depth
is "deep-enough" already). Focus attention on getting better features to get better performance.