7
$\begingroup$

I am trying to include a variable like "account number" which is an "ID" as a predictive variable for a logistic regression model. In fact there are several columns in my dataset that are "IDs" but are important in predicting the outcome. for example if an account number is associated with a fraudulent phone number, I want my model to capture this relationship. Transforming these variables to categorical variables is not a solution since the are over 20 millions of them in each column. I have done word embedding and transformed each ID to a vector space then applied PCA to reduce the size of the space and replaced every ID with a vector and an ID column to several columns which are now continues variables. However this idea fails when the number of IDs are over a million. I appreciate if you point out an example/study similar to my problem. and please let me know if I need to explain the problem in more details. I have used gensim library in python for word embedding and amazon ML to train and test the model.

Update: recently tensorflow developers added a new functionality called "hashing" and "embedding".

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ are there multiple rows with the same account number? $\endgroup$
    – Glen_b
    Commented Nov 6, 2017 at 22:55
  • $\begingroup$ yes, and an account number might be associated with different phone numbers for example. $\endgroup$
    – hm6
    Commented Nov 6, 2017 at 22:57
  • 3
    $\begingroup$ How are you dealing with the obvious source of dependence that results? Do you just ignore it and treat them as independent or is there some multilevel / hierarchical mixed effects type model here? (As a not completely separate issue, might you be looking to cluster/categorize the IDs in some way?) $\endgroup$
    – Glen_b
    Commented Nov 6, 2017 at 23:05
  • $\begingroup$ I have ignored the dependency so far. I am not sure how to cluster/categorize IDs. I have done some network analysis, mainly creating a cluster_Id for all directly/indirectly connected account and phone numbers. $\endgroup$
    – hm6
    Commented Nov 7, 2017 at 14:17
  • 1
    $\begingroup$ two online articles related to word-embedding and IDs. towardsdatascience.com/… omarito.me/word2vec-product-recommendations $\endgroup$
    – hm6
    Commented Nov 7, 2017 at 14:26

2 Answers 2

Reset to default
6
$\begingroup$

ID variables like phone number should not be included as predictors, because you are trying to train a model to understand general patterns. Phone number doesn't offer the model any real insight into what drives fraud vs non-fraud. You'd be better off using a lookup table after the fact to flag transactions linked to that phone number.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ many phone numbers and account numbers do not have a one to one relationship instead they are connected and form networks. the shape and changes in a network can be a signal to fraud. However, without IDs model can not understand their connectivity. $\endgroup$
    – hm6
    Commented Nov 7, 2017 at 14:04
  • 1
    $\begingroup$ ID variables that identify a particular training example should never be used as predictors. $\endgroup$
    – HEITZ
    Commented Nov 7, 2017 at 18:24
2
$\begingroup$

With new functionalities implemented in tensorflow such as hashing and embedding, I was able to take advantage of ID variables in my data and use them as predictive variables. You can find the explanation in this post: https://developers.googleblog.com/2017/11/introducing-tensorflow-feature-columns.html

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.