I am working on active learning and I was wondering about the difference if we split the dataset into training and testing or collecting and labeling the training and testing datasets separately. Either way, the ratio between training and testing will be maintained (70%,30%).
I want to select samples to label them and train the model to boost its performance and then I come back again to select new samples to label for testing because the model will be well trained and select hard and new examples to the model. Doing such that, is it appropriate?
Thank you for your answers.