6
$\begingroup$

In the scenario in which I'm working each entity could be represented in terms of N distinct properties that I will call p1, p2, ..., pn. For each of them, an entity, can have its specific range of values for a single property. An entity's property couldn't have any value associated to it.

Formally speaking, given an entity e and calling pk a specific property associated to e, it can have nk values associated (nk >= 0).

This is a brief example:

<entity 1>
director: Tarantino
distributor: Miramax
producer: Bender
starring: Fox, Hannah, Dreyfus, Thurman

<entity 2>
director: Tarantino
distributor: Miramax
producer: Bender
starring: Travolta, Jackson, Thurman

I'm not able to understand how to represent these entities in order to be used by a machine learning algorithm. I have seen some others that transform multi-valued categorical attributes in multiple binary attributes. I think that in this case is fundamental to analyze each property exactly as it is, with all its values.

After the creation of this method, I will be able to represent all the entities in my dataset. Supposing that, in this scenario, there are users that express a preference towards some of the entities (only a binary preference e.g., Tom likes entity 1 and so on). Using a machine learning algorithm, I would like to generate a model for each user in order to be able to understand if a new entity (a new film) could or not be liked by that user (this can be expressed with a value in a range between 0-1).

At this point I have two distinct questions that I will report here:

  1. What could be a proper way to represent the dataset entities?
  2. What could be a proper machine learning algorithm which is able to generate a model using the entities that I have?
$\endgroup$
4
  • $\begingroup$ What do you mean that you cannot represent the entities? You did give an example representation. So the question is how to represent it in as SQL data, or how to tabularize it (like with co-called dummy variables)? $\endgroup$ Mar 14, 2015 at 20:28
  • $\begingroup$ I've edited my question. What I don't know is which is the proper way in order to use this data in a machine learning algorithm. $\endgroup$ Mar 14, 2015 at 20:59
  • $\begingroup$ Do you have any machine learning algorithm in mind? (There are many, some have special restrictions.) $\endgroup$ Mar 14, 2015 at 21:03
  • $\begingroup$ Unfortunately not and this is due to the fact that I have multi-valued categorical attribute. I haven't find any algorithm which works in this situation. Do you have any idea? $\endgroup$ Mar 14, 2015 at 21:11

1 Answer 1

5
$\begingroup$

The most standard way of dealing with variables having an array of values is using dummy variables, i.e. creating a column for each possibility and assigning 0 and 1 depending if a n attribute is absent or present, respectively.

See for example how to do it in Pandas (if you are using Python) and Generate a dummy-variable in R.

The good thing is that you can treat 0 and 1 as categorical (e.g. for decision trees or random forests) or numerical (for various regressions, k-nearest neighbors, principal component analysis, k-mean, etc). Sometimes you need to convert all variables to numerical, even if there is only a single attribute per entry.

The bad thing is that if there are many options, either you need to restrict yourself to only the most common or perform some dimensional reduction with the principal component analysis.

The ugly thing is that even if you are using categorical-only variables, then you typically present single-valued variables with text/id, while multi-valued with dummy variables.

$\endgroup$
5
  • $\begingroup$ Thank you for your clear explanation. Could you please provide to me a simple ARFF representation of the entities? I ask to you this simply to have a good reference from which I can start my work. $\endgroup$ Mar 15, 2015 at 11:03
  • $\begingroup$ What is "ARFF"? $\endgroup$ Mar 22, 2015 at 13:48
  • $\begingroup$ The dataset file format that is used in the Weka library $\endgroup$ Mar 22, 2015 at 13:54
  • 1
    $\begingroup$ @AlessandroSuglia I am not using Weka, so I have no idea. But did you try googling Weka dummy variables? $\endgroup$ Mar 22, 2015 at 13:57
  • $\begingroup$ The dummy coding only works for independent variables, how about multi-level categorical dependent variables? $\endgroup$
    – raygozag
    Mar 7, 2023 at 2:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.