I have the following:
- Label (y): a boolean flag indicating something is good or bad
- Features (X): lower-level features that are believed to be correlated with the boolean flag. Some of them are continuous numbers and some are categorical.
Using these data, I need find a "magic formula" which calculates the "goodness score" of the sample based on the set of features (X). Despite I already have label and features for each sample, I need to figure out a numerical score which I could use to rank the samples by.
How should I go about designing/implementing this? If the label was a continuous number, I could simply use linear regression with L1 regularization, get coefficients, and multiply each feature by the weights to calculate the score. However, given the label is boolean, this becomes a classificiation problem and I'm wondering if there's a similar model I could use to suit my needs.
I know ensemble methods from sklearn provide feature_importances_, but I'm not sure if it's meant to be used to reconstruct a score by multiplying features. It does a great job telling me how much each feature is contributing to the decision process, but I'm not sure if it's meant to be used for my purpose.
Any help would be greatly appreciated!