I see two statistical problems here that can be worked around with a different approach.
The first is using a split sample for model building and evaluation. The risk with the split is that you lose both power in building the model and precision in testing the model. Frank Harrell estimates that you need at least 20,000 subjects to get around those combined risks. On that basis, you don't have enough cases to provide adequate power for the split-sample approach.
The second is how to apply the principles of non-inferiority testing here. That's usually done with an outcome model, to evaluate non-inferiority of some treatment versus a standard of care. The treatment effect is typically represented in a survival model by a coefficient that follows an asymptotic normal distribution, so that its standard error can readily be used to test for a pre-specified level of non-inferiority.
Here, the question is whether the new model type provides non-inferior predictions versus the well-established FRAX tool. If you were to evaluate with the Brier score (better than the C-index for model comparison),* I'm not sure that there's a theoretical form for the distribution of Brier scores in this context to allow for a corresponding parametric test.
Applying the optimism bootstrap should get around both problems. This uses all of your data to build the model while providing a reasonable evaluation of how your modeling approach would work when applied to a new sample from your underlying population, without depending on a parametric distribution for testing.
The idea is that bootstrap re-sampling from your data set mimics the process of taking your data set from the underlying population. Thus repeated model building on bootstrapped sample of your data and testing all of them on your full data set provides estimates of both bias and variance to be expected when building your type of model on multiple samples of the underlying population.
What's required is to repeat all of your modeling steps on each of the bootstrap samples: in your case, all the steps for setting up the DeepHit neural-network survival model. Then the distribution of Brier scores among the bootstrap-based models, all evaluated on the full data set, will show you how frequently a Brier score from your DeepHit modeling process will be worse than some pre-specified difference from the FRAX Brier score.
That said, you might be expected to deal with questions about whether this type of model will be better suited to clinical practice than the FRAX score, which is based on a few easy-to-understand risk factors and either a bone-density measurement or a body-mass index. Neural networks aren't noted for producing models that are easy for humans to grasp; see this discussion, for example, about the tradeoffs between machine-learning models and more traditional statistical models in practice. One of the big advantages of FRAX is how easy it is to explain to a patient as a basis for guiding treatment decisions. In contrast, even the model builder might be hard-put to explain why a neural network decided that a particular patient is thought to have a high risk.
*See the discussion here for example. The C-index only addresses discrimination in terms of rank-order of probabilities (maybe not very useful when most cases don't have events), while the Brier score helps evaluate calibration.
