2
$\begingroup$

Suppose I have a large data set with lots of features(attributes). And I'm tasked to build some kind of scoring model to rank certain objects with all these features. How do I go about doing this?

From my understanding so far, I like to think of this as a supervised learning problem. But the problem is there is NO labeled classes (or at least it's not apparent). How can I rank order these objects? The closest thing I can think of is credit scores, but in credit scoring models, one supposedly has labeled classes as to who historically was good and bad.

Should I invent/create some metric based on the list of attributes and use them as labeled cases? Like if attribute$_1 > x$ and attribute$_2< y$ etc., then it's considered "good"

I believe they want a numerical ranking (i.e., scoring all the objects have numerical scores assigned to the objects like credit scores). If that's the case, then do I even need machine learning/data mining? Can't I just rank it by these attributes once they agree what the ordering means?

Improve this question
$\endgroup$
2
$\begingroup$

If you have neither labels nor ranking examples, I don't know what you could do with your data other than clustering it based on similarity. The ranking function that you are supposed to learn can be a user's preferences (e.g. when I type "learning" in a search engine I prefer "machine learning" results rather than "e-learning"), a risk score for a bank (i.e. you would not be modeling the clients preferences, but the bank's), etc. That is, the set of possible rankings is $N!$, and there is not a universally good one.

In ranking, you usually have some examples of ordered objects. The task is to learn a ranking function that can be:

  • point-wise: you learn to score every item based on its attributes. The score is used for the final sort.

  • pair-wise: you learn to sort in pairs. You have examples like $A \succ B$, and then your function learns to make pair-wise decisions. Since if you put all the pairs together, you will probably have inconsistencies (e.g. $A \succ B, B \succ C, C \succ A$ ) it is your task to create a final maximal consistent ranking from these pairs.

  • list-wise: you try to learn a ranking function whose output will be a final list.

Point-wise and pair-wise are the most common ones since it is easier to rank locally rather than all items at once (list-wise).

The pointer to all this is "Learning to rank" (Information Retrieval) or "Preference Learning".

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks for answering. Today, I found out there is a RANKING function. They used percentile on several hand-picked features/attributes. And the aggregate score is a simple average. I still think there should be more intelligence to this(???!) My problem is that a high percentile does not necessarily translate to a good score. I think there needs to be labeled classes to make this work. Any thoughts? $\endgroup$ – Rachel Jul 16 '14 at 4:53
  • $\begingroup$ I think you should have labeled classes too. There are more intelligent ways to do the ranking, sure. Take a look at SVMrank. It uses a tuned SVM (Support Vector Machine) to rank items based on features (whatever you think are important) and labels (pairwise comparisons). $\endgroup$ – alberto Jul 24 '14 at 9:05
  • $\begingroup$ Here's another question. I have two data sets that can be potentially joined by some key(common elements). The problem vaguely stated is to find how certain events in set A is related to set B. Data is combo of categorical and numeric. It sounds like frequent item set problem(association rule mining)?? I know a bunch of DM methods but I'm struggling with mapping these somewhat vague biz requirements into the right models/mining techniques to apply. Ideas? Thanks! $\endgroup$ – Rachel Aug 5 '14 at 2:50
  • $\begingroup$ Set A is a large table with many fields. Set B is another large table with many fields. Potentially it's many tables joined together to form set A and set B. To put it more concretely, what data mining(DM) methods does one use to find how related certain fields in table A is related to fields in table B? Correlation is NOT the right technique. I'm thinking association rule mining. Hmm.. $\endgroup$ – Rachel Aug 5 '14 at 4:20
  • $\begingroup$ @Rachel, you should open a new question for this since it is unrelated to the original one :) $\endgroup$ – alberto Aug 12 '14 at 9:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.