I am working in the space of cancer statistics.
I am looking for reasons why it is important to validate statistical observations in an independent biological dataset.
Does anyone have a list or a reference to a good article about this topic?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
The basic idea is simple: you want to see if your model just works on the particular data sample from the particular population you were working with, or if it can be generalized to different populations, with different characteristics measured potentially in different ways, and treated at different institutions.
In practice, it's not so simple. Good starting points as references are the BBR course notes by Harrell and Slaughter, in particular section 10.11, and Harrell's RMS course notes in Sections 5.3 and 20.11. Those references provide some links to the literature.
As discussed in those references, there can be significant downsides to external validation. You might have to wait a long time to get enough completely external cases to test your model with any precision. You could need 200-400 events in the external data set, with 15 or so events per predictor in your model, to get a good enough external validation set. If you have information from multiple populations and treatment sites, including information about those populations and treatment sites in the model is probably a better approach, followed by rigorous internal validation via bootstrap resampling.
So the ideal of a completely separate data set is not always a realistic goal or the best way to proceed.
