I have some raw data that I would like to transform into a dataset, then ask for external parties to help me build model (with criteria such as minimising log loss, maximising area under curve).

  1. If I pass the dataset to an external parties, is it pretty unlikely that the dataset could be re-engineered back into raw data (which could be confidential for internal only)?

  2. The outsourcing is workable because I could pass a dataset to external parties. Is it fair to say that machine learning (and AI in general) would most likely need datasets for years to come? If one day there is no more dataset, I don't know how to outsource without keeping my confidential info.

Thanks in advance.

  • $\begingroup$ Is there a strong reason to suppose that non disclosure agreements are ineffective? I have been on both sides of this transaction and usually the contract has specified the confidentiality of the data. $\endgroup$ May 22 '16 at 10:47

I'm interpreting your question as how to mask your data-set before sharing it with external party (analytics vendor) for analysis; after the analytics has built a useful working model you would like to deploy it in-house to score/predict the required results on real data.

You could follow this approach:

  1. Organize your data-set into a fully normalized data model so that your "facts" table is composed of only foreign keys. For example, user attributes are replaced by a user-key number which references your internal table of users. Numeric attributes could be renamed as NUM1, NUM2, etc. and remove any units such as dollars/kg/etc. This is not as simple as it seems since the exact approach is based on the use case and business process the dataset is representing.

  2. Share the data-set and attributes with the vendor alongwith some hypothesis indicating how each attribute may be affecting the result. This would help the analyst understand the data. The vendor could then build the model using this training/test dataset and send this back for you to deploy it.

  3. While deploying the model in-house, with you would need to change all the anonymized attributes back to original form and re-train the model; or, transform your new data into the anonymized form for the model to score/predict.

A big disadvantage of this approach is that the analyst cannot add or enrich the data-set with external information which might improve the accuracy significantly. For example, usually a customer's address is removed to protect his/her privacy and Geo-spatial analysis may provide strong indicators of how a customer behaves. But since this information is lost while anonymizing the data-set, the resulting model turns out poorer than otherwise. Another disadvantage is that an experienced team of analysts would have applied certain insights or hypothesis to the problem and help reach a good model quickly, the team may not be able to apply this insight if most of the attributes are anonymized.

For your second query, assuming you need an external vendor to build machine learning models, you would always need to send training datasets to them in the future.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.