Data Fusion/Statistical Matching: Match Keys & Algorithms? I need to fuse multiple surveys using statistical matching, and would like to comply with any best practices.  I have studied multiple documents but -- despite the plethora of information -- it's unclear what the best practices are. Specifically...
First, what are the best match keys? I'm considering using region, gender and age for the critical variables (i.e.: MUST match) and persons in household, children, language and income for the non-critical ones.
Second, what's the best algorithm? I'm considering k-Nearest Neighbor.
Third, what should I do about mis-matching match keys? For instance, income brackets may differ in the two surveys.
Finally, what free tools exist?  I have Weka and R (and will look into StatMatch).  An ideal would be a tool that will produce a fused CSV from two (or more) CSVs and a set of match keys.
Also, I'm fine being pointed to a book. I'm a developer (not a statistician), so I'll happily read something geared to my level.
 A: So happy to see someone is facing the same problems as me!
I agree that a pletora of documentation exists about statistical matching, but still it is unclear what solution is the best. I think the approach depends a lot on the field of study, that is: official statistics, market research ... The references, tools & approaches  are quite different as long as I know. What is exactly your field?
You ask mainly  for three things: (i) best practices, (ii) tools, (iii) books- references. I’ll try to respond to those three in order:
Best practices
Independently of the field where you want to apply statistical matching, a unique best approach ("no free lunch for statistical mathing ") does not exist. I think the solution will depend highly on what type of problem you have, I mean:  if you have many common information or variables between the files but few specific variables (columns that you must fill); or if you have few common variables but a lot of specific ones ... I think the most useful way to think about statistical matching is that the final fused file (that is, a synthetic complete file that you will obtain, containing all the information for the original files) should correctly reflect the true relations on the underlying population – be it composed of individuals, houses, schools …  But the point is that, since you don’t usually have any record with values for all the variables in the files you want to match, you can never check if your algorithm is reproducing the ‘true’ values (i.e. techniques like crossvalidation in machine learning algorithms are not applicable here). That means that the validity is much, much harder to assess.  
Bayesian techniques can overcome this problem because they are ideal to deal with uncertainty, and I think they are the most promising approach, but I haven`t found any clear guide to proceed this way yet.  For the beginner (just as I am), I think the best way to start will be to try to use as much common information between the files in order to reduce uncertainty in the fused file. This is because the more the common variables (say Z) you use to determine the specific ones (say X, Y in the case of two files), the more you will reduce uncertainty. So you must convince yourself that a big part of the available time in your project must be spent in choosing good common variables. That means you need to compare distributions between the files (variables must measure the same thing in both files), and probably you will need to aggregate categories in order to make variables comparable. Also, you will probably need to distinguish between strata variables (matching is performed within those strata) and variables that will be used to search  for similar recipient/donor pairs. Also, for numerical variables, consider taking z-scores to make distributions comparable. And put a lot of imagination trying to construct new variables – for example using factor scores (constructed by factorial analyses).
Then you should use a good (robust) algorithm, and finally validate the fused file at least to make sure that the observed distributions in the original files (univariate-bivariate or higher) are correctly reproduced in the fused file (this is easy to do, and must be a minimum requirement). Ideally, you should also check that the fused file reflects correctly the true correlations in the populations (but this is much harder to do). For this purpose you could:


*

*Perform a simulation study: this means trying to simulate your statistical matching situation splitting one of the files randomly in two files, deleting some variables, applying your algorithm and checking if you obtain good results. This is a good thing to do in order to understand your algorithm better, and to ensure it is robust, but it doesn’t make sure the fused file reflects the true correlations, because you don’t really have that information anywhere.

*Try to use algorithms in a bayesian approach. That is: as long as you can, don’t use k-nearest neighbour or similars (frequentist approach) because although they can be good at reproducing marginal distributions, they are too limited to assess validity.


Tools
I only know about tools in R:


*

*StatMatch package for official statistics developed by ISTAT, based on frequentist techniques, but an interesting one because of its documentation and good hot-deck procedures (you mention this). It deals with survey weights, which is a serious problem with no easy solution

*mice package: implements multiple imputation algorithms applicable to matching. You can start by reading this.


References
The ones I said before plus:


*

*Statistical matching by S.Rässler. A very inspiring book (I think it is a must). 

*Some good guidelines + case studies for official statistics by Eurostat here.

