I am creating a support vector machine for extremely unbalanced data in which identifying instances of the rare class is of the highest importance. Since the data is so unbalanced, training and testing a model with no up-sampling results in an extremely accurate model that performs very poorly in terms of its true positive rate.
To ensure that the model is able to appropriately distinguish between the positive and negative class I split the data into a training and test set and performed up-sampling of the rare class within the training data. This allows me to use the test data to estimate the model's performance but it seems that it leaves me unable to utilize k-fold cross validation. Is the method I have utilized an acceptable approach? Is there another methodology that is recommended?