I am working on getting good probability from Random Forest algorithm for better decision making. Currently, I have trained the RF model with default parameters and then applied isotonic regression using CaliberatedCV sklearn library. Although I was able to get some improvement in terms of log loss, the brier score has not changed. I would like to request to help me find how I can get nearly perfect probability calibration for a multiclass problem from my Random Forest algorithm.
The dataset is highly imbalanced, therefore, I have set class weight parameter of RF to "balanced".