How many times should I repeat hold-out cross validation I am in the process of performing my first ever cross-validation. Yay.
Anyways, I am starting off with hold-out cross validation: I split it 50/50, and then I fit on one half, test on the other.


*

*Question 1: Is 50/50 an appropiate split?

*Question 2: How many times should I repeat this? 

*Question 3: If I do repeat it more than once, does it become just as "good" as stronger methods, such as k-fold cv?


EDIT: I have 3000 data lines.
 A: Answer 1: Probably not but there's no exact answer. Typically the test size is smaller than the training size. I may try to start with an 80-20 split and then experiment.
Answer 2: Again, there is no exact answer, it depends what you want to achieve.
You can use holdout validation to stop training when there is no improvement on the holdout anymore. In this case you don't need to define number of epochs.
Another way is to run multiple times and just take the model with  the best result. Number of reruns is an arbitrary choice.  
Answer 3: Repeating multiple times should improve results. Both random holdout and k-fold have pros and cons. Repeating random holdout can be as good as k-fold cross validation. The difference is how the data is split into training and validations sets.
+1: Don't forget to shuffle the data first.
A: 
Is 50/50 an appropiate split?

Depending on what you want to achieve with that split it may or may not be appropriate.

 How many times should I repeat this?

Sufficiently often, so that 


*

*all cases have been tested

*and you have either a sufficient number of surrogate models to show they are stable, or

*all cases have been tested at least 3 times, so that you can show the predictions are stable.

*If models/predictions are not stable, further increase the repetitions until model stability is not the concern for your performance estimate any more.



If I do repeat it more than once, does it become just as "good" as stronger methods, such as k-fold cv?



*

*It will stay more biased (compared to the performance of the model trained on the whole data set) as fewer cases are available for training.

*If you want to use the model you actually tested, you'll have to decide which of the many models you calculated. And those models will probably on average be worse than the model trained on the whole data set (again as fewer training cases are available).

*If you do sufficient repetitions as suggested above, the random uncertainty of the performance estimate will be as good as with any other resampling technique done properly.
