Hyper parameter tunning of stochastic models I struggle to find a clean answer for this question:
how to tune hyperparameters while the model does not produce similar results for different runs?
Before starting the tuning, I play around with the parameters to have consistent outcomes. However, during the tuning process, the outputs of the model become inconsistent for the given new set of the parameters. Therefore, the choice of the best parameter is not valid.
 A: This is a problem with a great many models including random forest, gradient boosted decision trees (e.g. XGBoost, LightGBM etc.) and neural networks. However, it does not make the choice of best parameters based on e.g. cross-validated performance invalid.
It makes the feedback you receive noisy, which you can try to reduce by either choosing some of the hyperparameters manually to reduce the noisiness (e.g. more trees in RF, more trees + lower learning rate for XGBoost etc.; doing this is usually a good idea) or by repeated model fitting for the same parameters (which you can also do at inference time by averaging the results of multiple models aka "random number seed averaging").
Alternatively, you can just live with the noisiness and be aware that you may or may not have picked optimal parameters (+ take it into account e.g. by doing a surrogate model on the hyperparameter space that allows for a noisy outcome to predict good hyperparameters to try).
In any case, of course, hyperparameters you somehow choose are not optimal (even when the performance assessment does not vary due to a stochastic algorithm), because there is some inherent randomness in how you e.g. do your splitting for cross-validation, there's limited data and that's not even talking about whether the choices will perform well on new unseen data in an actual use case.
