In machine learning every algorithm has a set of hyperparameters which needs to be optimized for best prediction performance. The simplest method for this optimization is called grid search which means to try all possible parameter value combinations. In this way one can find the best parameter values.
However, this does not give any insight about the interaction of parameter values. For example suppose that we have 5 hyperparameters p1, p2, p3, p4, and p5. There might be these kind of facts: as the value of p1 increases given that p3's value is low prediction performance increases. However, if p3's value is high then p1's value has no effect. There might be many more interesting facts like these. Is there any method for finding these kind of facts?