2
$\begingroup$

Despite the fact that this problem is probably overdone and heavily researched, I want to use machine learning to solve problems related to basketball (predicting which team will win the game, will a certain shot go in, etc.). To do that, I want to encode information about the line-ups, because ultimately the starting line-up for both teams will have a huge impact on a team's performance and ultimately their ability to win a game (eg. Cavs' chance of winning a game is significantly lower without Lebron, and much higher if Kevin Durant is not playing).

Given the player name (or some unique id to identify the player) is a categorical variable, how do I encode this to fit the appropriate format for prediction tasks? Ideally, I do not want to create a ton of binary variables for each player in the NBA (which represents whether the given player will play or not). I was thinking of tackling this problem with Python/sci-kit learn, which I usually feed in numpy arrays. Is it possible to modify my data to still use numpy/sci-kit for the prediction task?

Similarly, another variable would be the opponent team (LAL, GSW, BOS, etc). There are significantly fewer choices than the number of players of course, but still 29 which is a considerable amount. While I could create 30 binary variables, is there another option? If I match each team to a number, is that legitimate (If so, are there any downsides? If not, can you expand on why? I am somewhat reluctant because it is unclear what these numbers would actually represent.)

I have read a few articles about encoding categorical variables, and it seems like a fuzzy area. I did not see any clear/obvious answers and could always use the guidance and expertise. If you have any ideas, suggestions, or relevant videos/blogs/papers, please let me know.

| cite | improve this question | | | | |
$\endgroup$
  • $\begingroup$ Depends on the ML-algorithm (and it's assumptions). Encoding it on the number-line is obviously killing linear-models. $\endgroup$ – sascha Jun 7 '18 at 22:06
  • 3
    $\begingroup$ There are a few ways to reduce the dimensionality of the encoding. One is the feature hashing (implemented in sklearn) which will lead to some 'collisions' between players. If you think about using neural nets, you can read about the Embedding layer (e.g. in Keras). This will transform your out to a fixed length vector. Players with 'similar' performances or whatever will be closer in that vector space. $\endgroup$ – Stergios Jun 8 '18 at 8:04
  • 1
    $\begingroup$ I think you need to be more specific about your intended meaning of ML algorithms. If you are referring to NNs, my understanding is that they require continuous features and are, therefore, unable to treat categorical features qua categorical. This means that the levels of any categorical feature have to be transformed into a set of 0,1 dummy variables. There is literature that deals with truly massive categorical data such as 5-digit zip code which contains roughly 35,000 residential levels, e.g., bear.warrington.ufl.edu/centers/MKS/abstracts/vol22/no1/… $\endgroup$ – Mike Hunter Jun 8 '18 at 13:56
  • $\begingroup$ Thanks for pointing out the ambiguity. I was thinking traditional classification algorithms at this point (logistic regression, random forests, svms). While I am open to NNs, I am less familiar with them and therefore was not considering them right now. $\endgroup$ – Jane Sully Jun 8 '18 at 15:18
3
$\begingroup$

I've seen feature hashing and embedding mentioned in comments. Apart from that you can try clustering players by IDs if you have some additional data.

Another approach which is suitable for categorical data with many level is mean encoding.

Mean encoding (also sometimes called target encoding) consists of encoding categories with means of target (for example in regression if you have classes 0 and 1 then class 0 is encoded by mean of response for examples with 0 and so on). There are some answers on this site on that which provide more detail. I also encourage you to see this video if you want to get more about how it works and how you can implement it (there are several ways that to do mean encoding and each has its pros and cons).

In Python you can do mean encoding yourself (some approaches are shown in the video from the series I linked) or you can try Category Encoders from scikit-learn contrib.

| cite | improve this answer | | | | |
$\endgroup$
  • 3
    $\begingroup$ Another variant of target encoding is also available in the dirty_cat package (dirty-cat.github.io) which is on research about learning on dirty categories (disclaimer: I am an author of the package). $\endgroup$ – Gael Varoquaux Nov 27 '18 at 23:58
  • $\begingroup$ @GaelVaroquaux interested package. I would like to try it. $\endgroup$ – Yong Wang Sep 16 '19 at 13:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.