I have access to two large medical datasets of observational records in the UK. The first - Clinical Practice Research Datalink (CPRD) - has data on 100,000's of patients - largely dates of doctor's visits, diagnoses & prescriptions. Crucially, it has little information on hospitalisations.
The second dataset is of hospitalisations - Hospital Episode Statistics (HES) - and importantly, I can identify HES patients in the CPRD. The HES data is a lot smaller - perhaps 5% of CPRD patients (I haven't the figures to hand)? - but it's still tens of thousands of patients.
The brief is to look for signals in the CPRD data that might indicate a hospitalisation - using the HES data to verify this. I'll likely do this within a cohort of patients with a specific illness (respiratory disease, for example) which should make their medical patterns more homogeneous than the general population.
The reason for the research is that CPRD is richer in medical history making it the most useful - but lacks that crucial element of hospitalisations. Still, it does have some - a patient might visit the doctor's and be told to go to ER, for example - this will be recorded in CPRD.
Ultimately, I'm not optimistic I'll find much but I need to give it a good go. My background is traditional frequentist statistics where tools such as agreement measures just seem woefullly inadequate. I have a nodding knowledge of some bayesian and machine learning techniques - textbook rather than practical - and the latter seems appropriate in this case.
I'm really looking for keywords & ideas to research (not necessarily full answers) as this will help greatly.
Also, I'll likely program this in SAS 9.1.3 (or 9.3 eventually) - if that makes any difference - though I might be able to use R (though dataset size would definitely be an issue).
I'm grateful for any help.
Resp Arthur Small: Yes, I suppose you could use CPRD records in that way, using HES records to 'confirm' hosps. I wonder though - given that CPRD is longitudinal data - at which points in CPRD you determine these probabilities (without first peeking at HES to get the date). Still, it definitely gives me something to investigate further - and is in my world of inferential stats!
Resp Peter Ellis: It is possible to use CPRD patients with HES (and the overlapping period) so yes, we can be sure that all hosps are recorded in HES for that subset of CPRD patients.
I can't see how logistic regression would work however. If a patient has a HES hosp in June and Nov of a given year, I'm not sure how regressing on CPRD events will point to 1 hosp and not the other. I think it's the time element that's confusing me. I 'anticipate' a cluster of CPRD events (and event types) around HES hosps - it's that sort of analysis I believe I need. Thanks.