Privacy preserved record linkage What is the best way for privacy preserved record linkage in data mining.  I am final year university student.  My thesis topic is related to data mining.  I am working with data and I need to privacy preserved record linkage.  I used talend, pentaho software.  But I don't know what is the best software or best way.   
 A: Different countries have different privacy requirements and so from a legal perspective, so you should verify any legal requirements with a lawyer that practices in your jurisdiction.  That being said, it is often times enough to simply remove any identifying information from your dataset (e.g. name, phone number, email address, mailing address, social security number, etc.), especially if it's not being used for analysis, and to then keep it in a separate, encrypted dataset all together, linked by randomly generated ID's which are maintained on the analysis dataset.  This is important if, for example, there is a requirement to preserve the linkage (i.e. sometimes this might be required if there is a chance that you'll need to notify a patient of some health condition that can only be uncovered during analysis of genetic data, for example).  If there is no need to preserve a linkage, you may not even have to keep an identifying dataset at all.
For example, say you have a survey dataset that has the following form:

Where Q1-Q3 are substantive answers to a set of survey questions and everything to the left of those variables/columns is potentially identifying information.  To de-identify this dataset, then, with one approach, the first step is to generate a random unique ID for each individual/observation/row in your dataset as shown below:

Next, you'd need to split the dataset into two files.  The first one consists of the ID variable along with the identifying variables either alone (e.g. name if it includes first and last) or those variables which combined could be identifying (e.g. First Name, combined with Date of Birth, Gender, and Zip Code might be considered identifying in certain areas -- especially in zip codes where race is not ethnically diverse and could lead to identification).  The second one would consist of that same ID and the variables that do not contain personally identifiable information (either singly or when combined with out variables).  For example, let's say that you've determined that in the area where your analysis applies, there are a handful of African Americans (i.e. Race="B").  Let's further suppose that there is only one female (Gender="F") African American.  Then it would be appropriate to remove at a minimium one of gender, race and also zip code from the dataset, along with any other identifying information (in this example let's assume that this also includes DOB and name).  Then the identifying dataset which would be encrypted and preferably kept offline and completely separate from the analytical dataset (and if possible controlled by someone who is not doing the analysis) would look like this:

and the analytical dataset would look like this:

Note how the ID is contained in both files and all information that can lead to disclosure of any particular individual is essentially quarantined into the identifying dataset.  The key here it to assess the risk of disclosure of uncovering who a record belongs to based on demographic or other information contained in the record.  This requires an understanding of the population that you are trying to analyze in addition to understanding the information contained in your dataset.  You must be mindful that (in this example) even answers to certain survey questions themselves, aside from demographic information, might lead to inadvertent disclosure of an individual's identity as well.  
Of course there are many other methods that exist for handling these kinds of issues (try researching "synthetic datasets" for example and you'll see quite a bit of work on statistical methods that attempt to limit disclosure risk through things like propensity score modelling, imputation, etc.  But this should get you started in the right direction.
