I am comparing the performance of two regression algorithms on a same dataset. I want to use t-test to evaluate whether there is a significant difference between the performance of the two algorithm.
Say there are 200 samples in total.
What I am doing right now is using a leave one out cross validation to calculate the error of each algorithm. I run the two algorithms on the 199 training samples and test the fitted regression models on the testing sample respectively for each of the 200 splits of the dataset. So I can have 200 testing squared errors for each algorithm.
At last I run t-test to compare the two 200 squared error samples.
Is there any problem with this method? Or are there any better ways to do the comparison?
I will really appreciate it if any formal reference can be provided for this specific problem.