2
$\begingroup$

I am trying to include a variable like "account number" which is an "ID" as a predictive variable for a logistic regression model. In fact there are several columns in my dataset that are "IDs" but are important in predicting the outcome. for example if an account number is associated with a fraudulent phone number, I want my model to capture this relationship. Transforming these variables to categorical variables is not a solution since the are over 20 millions of them in each column. I have done word embedding and transformed each ID to a vector space then applied PCA to reduce the size of the space and replaced every ID with a vector and an ID column to several columns which are now continues variables. However this idea fails when the number of IDs are over a million. I appreciate if you point out an example/study similar to my problem. and please let me know if I need to explain the problem in more details. I have used gensim library in python for word embedding and amazon ML to train and test the model.

Update: recently tensorflow developers added a new functionality called "hashing" and "embedding".

$\endgroup$
  • 1
    $\begingroup$ are there multiple rows with the same account number? $\endgroup$ – Glen_b Nov 6 '17 at 22:55
  • $\begingroup$ yes, and an account number might be associated with different phone numbers for example. $\endgroup$ – hm6 Nov 6 '17 at 22:57
  • 3
    $\begingroup$ How are you dealing with the obvious source of dependence that results? Do you just ignore it and treat them as independent or is there some multilevel / hierarchical mixed effects type model here? (As a not completely separate issue, might you be looking to cluster/categorize the IDs in some way?) $\endgroup$ – Glen_b Nov 6 '17 at 23:05
  • $\begingroup$ I have ignored the dependency so far. I am not sure how to cluster/categorize IDs. I have done some network analysis, mainly creating a cluster_Id for all directly/indirectly connected account and phone numbers. $\endgroup$ – hm6 Nov 7 '17 at 14:17
  • 1
    $\begingroup$ two online articles related to word-embedding and IDs. towardsdatascience.com/… omarito.me/word2vec-product-recommendations $\endgroup$ – hm6 Nov 7 '17 at 14:26
2
$\begingroup$

ID variables like phone number should not be included as predictors, because you are trying to train a model to understand general patterns. Phone number doesn't offer the model any real insight into what drives fraud vs non-fraud. You'd be better off using a lookup table after the fact to flag transactions linked to that phone number.

$\endgroup$
  • 1
    $\begingroup$ many phone numbers and account numbers do not have a one to one relationship instead they are connected and form networks. the shape and changes in a network can be a signal to fraud. However, without IDs model can not understand their connectivity. $\endgroup$ – hm6 Nov 7 '17 at 14:04
  • $\begingroup$ ID variables that identify a particular training example should never be used as predictors. $\endgroup$ – HEITZ Nov 7 '17 at 18:24
1
$\begingroup$

With new functionalities implemented in tensorflow such as hashing and embedding, I was able to take advantage of ID variables in my data and use them as predictive variables. You can find the explanation in this post: https://developers.googleblog.com/2017/11/introducing-tensorflow-feature-columns.html

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.