I validate usage of a clinical cardiovascular score to predict the risk of dementia using data from a longitudinal study. Therefore, my outcome is binary (dementia yes or not) and the independent variable (the score) is continuous, of course I have a whole set of covariates.
I did Cox analysis to assess an association between baseline values and the outcome over time but now I would like to validate the use of the score. I thought about taking a random sub-sample of my cohort to split in training and test and run some sort of validation statistics (i.e. ROC curves) but I have some concerns about this for a number of reasons:
- My sample is relatively small ($n=2500$), and I am afraid that taking a sub-sample would reduce the power too much.
- Not sure whether the ROC (or alternatively the somerset) are the best tests in this case, as other tests (like those used in screenings evaluation) may suit better.
How shall I evaluate the use of this score? Can you suggest tests that suit better for the problem?
For data analysis I use Stata.