3
$\begingroup$

I have a formulation of a statistical problem in mind and haven't been able to find any literature/references about it. As professors that I asked also couldn't help, I thought I'd ask here.

Consider the problem of performing statistical analysis on patient records from several hospitals. For example, we want to determine whether some medication is effective for treating a particular disease. In a case where all the hospitals involved use the same type of patient record system, we just merge the data tables together and perform the relevant type of analysis.

Now consider the case where data structure differs among hospitals, e.g. two different hospitals have different sets of measurements about the patient. Some variables in a table of first hospital cannot be found in a table of a second one, and vice versa. We have explicit knowledge about how these sets correspond to each other, for example, we know that variable systpres in the first hospital's data set corresponds to pressure_systolic in the second hospital's data set. How do we do inference?

  1. Find what variables are available in both tables and disregard all other information. Merge 'cropped' tables. - This way we lose useful data.

  2. Merge original tables and try to impute all missing information. - This way we don't use the fact that the nature of 'missingness' is known and explicit knowledge about what data will be missing where is available.

Is it possible to somehow use the knowledge about data structure correspondence in a statistical analysis? Has that type of problem been solved somewhere?

Any pointers/ideas could be useful!

Thank you.

Improve this question
$\endgroup$
7
  • $\begingroup$ Could you be a bit clearer about the "explicit knowledge" that imputation would ignore? $\endgroup$
    – Scortchi ♦
    Commented Mar 7, 2014 at 9:28
  • 1
    $\begingroup$ Assume that you perform two regressions for two hospitals. For one regression you have factors $x_1, x_2, x_4$ (and weights $\beta_1, \beta_2, \beta_3$), for the second $x_1, x_3, x_4$ (weights $\gamma_1, \gamma_2, \gamma_3$). Why don't you just fit two regressions with shared weights? Fit them jointly by introducing the constraints: $\beta_1 = \gamma_1$, $\beta_3 = \gamma_3$. $\endgroup$
    – Dmitry Laptev
    Commented Mar 7, 2014 at 9:48
  • $\begingroup$ Scortchi, I've updated the question. Does that clarify? $\endgroup$
    – martinthenext
    Commented Mar 7, 2014 at 22:08
  • $\begingroup$ Dmitry, yes, that sounds like a good idea for linear regression. Also, I can think of some interesting adjustments to well-known algorithms to perform other tasks like clustering or classification in the described case. What I wanted was to find out the name of this class of problems and maybe a reference to some work in this area. $\endgroup$
    – martinthenext
    Commented Mar 7, 2014 at 22:12
  • 1
    $\begingroup$ Could you explain what you mean by "data structure correspondence"? It sounds like you mean that systpres is the same as pressure_systolic. But since you say we know those correspondences, I can't see any reason to impute any data values. On the other hand, if you mean Hospital A records a variable that Hospital B does not, what correspondence are you referring to? Or are you referring to automatically matching up those two fields with different names but the same data? Sorry, but I'm confused. $\endgroup$
    – Wayne
    Commented Mar 7, 2014 at 22:32

1 Answer 1

Reset to default
1
$\begingroup$

Hello Martin,

Here are my thoughts about the problem:

  1. Certainly, treat the variables like "systpres" and "pressure_systolic" as the same (and give them the same name to avoid confusion).

Concerning the remaining variables which are distinct in different hospitals, in general it's hard to find the answer, but, depending on the background knowledge and the nature of your problem, it may be possible.

  1. Construct new 'score variables'-weighted linear combinations of the variables (possibly normalized to the same range) which are unique for each hospital to obtain a 'score'-one new variable which will summarize what you want to compare. This is similar to credit scoring of debtors in a bank. However, this is where your background knowledge should be applied, since rapidly different combinations of variables will give very different results, rendering scores non-informative. It may also be a good idea to include variables of the same type in one score, i.e. only the count variables or only the continuous ones, etc.

  2. In case of OLS regression with manual variable selection for each hospital, compute correlations of the unique variables with shared variables for each hospital or try to understand how they're dependent. (I.e. in Dmitry's example, x_2 with x_1 and x_4 for hospital 1, and x_3 with x_1 and x_4 for hospital 2). If x_2 is highly correlated with x_1 and x_3 is highly correlated with x_4, you don't need to care about them and can safely compare using only x_1 and x_4 for the hospitals.

  3. Try to find such shared variables which approximate the unique ones well (i.e. proxy variables) (again, separately for each hospital). For this you can run regression of some shared variables on the unique ones and compare regression fit. If possible, in each hospital try to find the same proxy for as many unique variables as possible, then compare the proxies. Or ask for further observations of new proxies, which must be the same across hospitals. Methods to look up in relation to proxies and their estimation are:

They are all well described, for example, in http://www.ozon.ru/context/detail/id/3608597/ or in W.H. Greene "Econometric Analysis"

  1. Although I am not an expert on ANOVA, the problem very much reminds me about Unbalanced Incomplete Block Designs, or Unbalanced Factorial Designs, depending on the nature of the variables. See references in Montgomery's book "Design & Analysis of Experiments". In 5ed. it's pages 600-604.

Hope you will find this useful.

Regards, Vasily

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.