# To carry out a T test for two tests of data from one patient using the same metric do I have to assume the differences is normally distributed?

My supervisor told me that to carry out a t-test on two different models to see which one is statistically better I needed to carry out a normality test (or just find if the data is normally distributed by graphical means) on the difference between the two data sets.

There are two data sets, Model A and Model B which calculated their accuracy using the same metric from the same patient just using two different Machine Learning models ( A and B ).

Would I find the distribution of the difference between the two models? Or do I have to see if each model is normally distributed before satisfying the assumption to carry out a Paired T Test?

Sorry If its confusing!

• Do I understand correctly, that you have for each patient three values: a value computed by model A, a value computed by model B, and the true value? And then you want to know for which model, A or B, the difference to the true value is on average smaller? Commented Jul 20, 2022 at 11:50
• So for each patient, there's a value for Model A, A value for Model B and then I calculated the difference between the two models. @frank Commented Jul 20, 2022 at 14:22

for each subject you have 2 predictions, one for each model, so you calculate errors as error_A = target_outcome - predicted_outcome_A and error_B = target_outcome - predicted_outcome_B, and the difference is just error_A - error_B (one number per subject), which you can then plot or run the t-test on or additional normality tests