# What data mining/machine learning approach to use for a scoring model?

Suppose I have a large data set with lots of features(attributes). And I'm tasked to build some kind of scoring model to rank certain objects with all these features. How do I go about doing this?

From my understanding so far, I like to think of this as a supervised learning problem. But the problem is there is NO labeled classes (or at least it's not apparent). How can I rank order these objects? The closest thing I can think of is credit scores, but in credit scoring models, one supposedly has labeled classes as to who historically was good and bad.

Should I invent/create some metric based on the list of attributes and use them as labeled cases? Like if attribute$_1 > x$ and attribute$_2< y$ etc., then it's considered "good"

I believe they want a numerical ranking (i.e., scoring all the objects have numerical scores assigned to the objects like credit scores). If that's the case, then do I even need machine learning/data mining? Can't I just rank it by these attributes once they agree what the ordering means?

If you have neither labels nor ranking examples, I don't know what you could do with your data other than clustering it based on similarity. The ranking function that you are supposed to learn can be a user's preferences (e.g. when I type "learning" in a search engine I prefer "machine learning" results rather than "e-learning"), a risk score for a bank (i.e. you would not be modeling the clients preferences, but the bank's), etc. That is, the set of possible rankings is $N!$, and there is not a universally good one.

In ranking, you usually have some examples of ordered objects. The task is to learn a ranking function that can be:

• point-wise: you learn to score every item based on its attributes. The score is used for the final sort.

• pair-wise: you learn to sort in pairs. You have examples like $A \succ B$, and then your function learns to make pair-wise decisions. Since if you put all the pairs together, you will probably have inconsistencies (e.g. $A \succ B, B \succ C, C \succ A$ ) it is your task to create a final maximal consistent ranking from these pairs.

• list-wise: you try to learn a ranking function whose output will be a final list.

Point-wise and pair-wise are the most common ones since it is easier to rank locally rather than all items at once (list-wise).

The pointer to all this is "Learning to rank" (Information Retrieval) or "Preference Learning".

• Thanks for answering. Today, I found out there is a RANKING function. They used percentile on several hand-picked features/attributes. And the aggregate score is a simple average. I still think there should be more intelligence to this(???!) My problem is that a high percentile does not necessarily translate to a good score. I think there needs to be labeled classes to make this work. Any thoughts? – Rachel Jul 16 '14 at 4:53
• I think you should have labeled classes too. There are more intelligent ways to do the ranking, sure. Take a look at SVMrank. It uses a tuned SVM (Support Vector Machine) to rank items based on features (whatever you think are important) and labels (pairwise comparisons). – alberto Jul 24 '14 at 9:05
• Here's another question. I have two data sets that can be potentially joined by some key(common elements). The problem vaguely stated is to find how certain events in set A is related to set B. Data is combo of categorical and numeric. It sounds like frequent item set problem(association rule mining)?? I know a bunch of DM methods but I'm struggling with mapping these somewhat vague biz requirements into the right models/mining techniques to apply. Ideas? Thanks! – Rachel Aug 5 '14 at 2:50
• Set A is a large table with many fields. Set B is another large table with many fields. Potentially it's many tables joined together to form set A and set B. To put it more concretely, what data mining(DM) methods does one use to find how related certain fields in table A is related to fields in table B? Correlation is NOT the right technique. I'm thinking association rule mining. Hmm.. – Rachel Aug 5 '14 at 4:20
• @Rachel, you should open a new question for this since it is unrelated to the original one :) – alberto Aug 12 '14 at 9:13