I have thousands of observations and 20+ characteristics (way more if you transform them from categorical to binary characteristics). Is there some method that can be used to build a profile of the typical record of a certain class? For example, if country of origin and industry are two characteristics, is there a statistical method that I can use to determine that records in class A are typically from country X and business Y, but only if that relationship truly exists?
I figure I could use something like a logistic model with binary variables and see which variables have a strong predictive value, but I was hoping to learn some new tricks. Is something like the Kullback-Leibler divergence applicable here? A random forest did not seem appropriate because a single variable can have different directional impacts at different depths of the tree.