3
$\begingroup$

What would be a good way to obfuscate sensitive information and store e-commerce transaction data, to later perform fraud analysis on it. One idea that crossed my mind, was to hash each sensitive field with a hash function (e.g. murmur3 128 bit) and store. As an example if we had a column account holder name with a value John Doe, the hash operation produces some 128 bit value for John Doe. Due to the property of statistical randomness introduced by the hash algorithm, combined with the cascading effect, would this affect the property of Linear separability of the underlying data?

EDIT: Following up from the helpful comment by @AlexeyGrigorev. I do understand that just hashing the name would not contribute to obfuscation. The data I am preparing at the moment for my academic interests, has lot more sensitive information (fields like card info, etc ). I have all fields in the data hashed with the same hashing function. The hashed data is now has score of 0.71 and AUCROC of 0.75 with best tuned SVM (tuned RBF kernel). As my hashing function introduces a random distribution of the data in a $2^{128}$ space, so I am guessing it should have affected the linear separability of the underlying data. Correct me if I am wrong or wandering in offshoot irrational directions.

|cite|improve this question
$\endgroup$
  • $\begingroup$ I don't think the name would be a particularly useful feature anyway, so hashing it shouldn't make it worse. But speaking of obfuscation, just hashing the name may not be enough to truly anonymize the data set $\endgroup$ – Alexey Grigorev Jul 12 '15 at 10:58
  • $\begingroup$ How is your RBF kernel dealing with 128 bit data? $\endgroup$ – Memming Jul 12 '15 at 13:10
  • $\begingroup$ @Memming I actually followed the process described here and then I used Scaling to scale it down, before feeding into SVM. $\endgroup$ – Segmented Jul 14 '15 at 0:00
3
$\begingroup$

Using a hashed representation in a vector space is not good unless a locality sensitive hashing is used. In your case, you are using one that is intended as a non-cryptographic hash (murmur3), so it might preserve some locality. (It's good that you aren't using a cryptographic hash function!) This means that similar points in your original space is less similar in your hashed representation, and joint information between the entries might get destroyed.

In general, such hashing can destroy linear separability easily.

I believe it is very challenging to design a privacy sensitive learning where all entry has to be obfuscated. I suggest pre-computing a similarity/dissimiliarty of the raw data to find a obfuscated representation that preserves those in your final dataset.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ @Memmming Interesting edit. Thanks for the clear explanation. I did some lookup and found I was using LSH (unknowingly). Another interesting approach for obfuscation could be Homomorphic encryption on data and then analysing it. I tried to initiate a discussion on the topic here. Although the feasibility of such an approach would then be subject to research. $\endgroup$ – Segmented Jul 15 '15 at 18:08
0
$\begingroup$

It's interesting question, look at this how vowpal wabbit use hashing in linear models:

https://github.com/JohnLangford/vowpal_wabbit/wiki/Feature-Hashing-and-Extraction

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.