I am working on a medical data set of 60 test samples and I have the decisions of 45 medical experts for them over 5 diagnostic classes. How should I do hypothesis testing to show that the classifier I trained is not statistically very different than the group of experts with some significance level?
I basically have :
- Diagnosis by medical experts : 60 x 45
- Diagnosis by my classifier : 60 x 1
- Ground truth : 60 x 1
Solutions that I can think of:
Calculating the accuracies of both mine (1) and the experts (45). Then doing ztest with the mean and standard deviation of accuracies of experts to show that my score is coming from the same distribution.
Doing McNemar's test between mine and each of the experts and then saying 'we cannot reject the hypothesis of my algorithm is not different from ,i.e., 20 expert.
I am not really sure how to do this analysis scientifically. Basically I want to show that my algorithm is as successful as one of those experts in this task. Any other recommendation is also welcomed.