I have a naive Bayes classifier with two classes (target and non-target) and distributions for a number of features (the same for both classes).
I know that some features contribute more, or less to the overall classification accuracy. I can use KL-divergence to measure the information gain from feature distributions, but how could I use it to weight features?
Say I have features f1, and f2. I have a KL value for the two features and the class.
$KL(f_1, C_1), KL(f_2, C_1)$
$KL(f_1, C_2), KL(f_2, C_2)$
Where do I go from here?