0
$\begingroup$

For a finite population I have two data sources A and B with variables (X, Y) and (X, Z), respectively. The first data source is actually a full sample, the second is a simple random sample. The items in B cannot be identified precisely using X alone.

I want to combine these data sets using the common variable X, assuming conditional independence of Y and Z given X in the simple case and perhaps later using auxiliary information on the interdependence of X, Y and Z. The resulting data should be an extension of the full sample A.

+---+---+
|   B   |
+---+---+---+
    |   A   |
+---+---+---+       +---+---+---+
| Z | X | Y |       | X | Y | Z |
+---+---+---+       +---+---+---+
|   | x | A |       | x | A | B'|
| B | x | A |       | x | A | B'|
|   | x | A |       | x | A | B'|
|   | x | A | ----> | x | A | B'|
|   | x | A |       | x | A | B'|
| B | x | A |       | x | A | B'|
|   | x | A |       | x | A | B'|
+---+---+---+       +---+---+---+

From the literature that treats data fusion and statistical matching (D'Orazio et al. 2006, Kiesl and Rässler 2006) I understand that my problem does not fall in this category, since A and B are not disjoint. Kiesl and Rässler call this "database enrichment" (case 2 in Figure 1, page 4), but a search for this term didn't find anything useful.

Is there a more specific name for the task I am facing? Could you point me at literature?

$\endgroup$
1
$\begingroup$

Edit: I'm no longer convinced that I wrote below is not actually correct, as I missed the line "the items in B cannot be identified precisely using X alone." The diagram makes it look like for some observations you know all of X, Y, Z, but maybe that's not the case?

This is a missing data problem (you are missing the variable $Z$ for most observations), and what you are trying to do is called imputation. I don't know exactly what the context of your problem is, but you should be aware that frequentist single-imputation methods tend to underestimate uncertainty, so if you want to do hypothesis tests or confidence intervals then you should look at multiple imputation (imputing each missing value several times, resulting in several data sets) or missing data estimation within a Bayesian framework.

$\endgroup$
  • $\begingroup$ I never observe (X, Y, Z). Sorry for the confusing diagram, I tried to fix that. $\endgroup$ – krlmlr Dec 4 '12 at 18:33
  • $\begingroup$ It still kind of looks like you observe (X,Y,Z) sometimes. I would represent the left side of the arrow as two different data tables, probably. But that's not terribly important. I'll think about it some more. $\endgroup$ – Jonathan Christensen Dec 4 '12 at 18:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.