How to report neural network training results

Question

I'm writing on my first scientific paper and I'm a little bit lost how to report my results: I have trained a neural network to identify a disease risk group from genetic data. In a monte-carlo simulation the results of my training processes are a little bit unstable over different training \ test samples:

Accuracy:
Min.  1st Qu.  Median    Mean   3rd Qu.   Max. 
0.47  0.69     0.77      0.76   0.83      0.94 

ROC-AUC:
Min.  1st Qu.  Median    Mean   3rd Qu.  Max. 
0.29  0.57     0.79      0.72   0.88     0.96 

Overall I'm very happy with the results but how should I report those results? Should I discuss a single model \ test \ training sample combination more closely? If so, which? Do you have tips, what to avoid in any case? Thanks in advance!

Answer 1

One way to report the result would be to perform cross-validation, and report min, max, standard deviation and average. If you compare your results against some other method, you can use some significance test such as approximate randomization. The performance analysis in the paper could try to explain why the train/test impacts the results more than expected.

---

As a side note:

• many papers unfortunately only report one number (e.g., "F1-score = 0.65").
• in addition to the train/test split, different runs with different initialization might have quite some impact on the neural network's performance (e.g. see https://arxiv.org/abs/1603.03827 table 3)