I am working on a frog classification problem where I have 15 unique species and 2 recordings per species. Would it be a valid experiment if I were to have my training set of the 15 unique species with 1 recording per species and augment it through, time-shift, pitch adjustment, noise injection, spectral augmenting, and other augmenting techniques? And although my test set is also very small, it is a good representation of the data because it is 1 recording per all species.
$\begingroup$
$\endgroup$
2
-
$\begingroup$ Can you classify the species into higher groups like genus or some other attribute? That would increase the n per group $\endgroup$– neriticzoneCommented Jul 26, 2023 at 22:41
-
$\begingroup$ Keep in mind that the most basic task that is anywhere related to your goal is the estimation of a single proportion. This requires n=96 to have any reasonable margin of error. $\endgroup$– Frank HarrellCommented Oct 17, 2023 at 12:19
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
This is known as a few-shot or one-shot classification problem, which is a widely studied problem setup, also in audio machine learning. The key to this task is generally to have a strong data representation, where the classes of interest is easily separable. Usually this is done using a pre-trained neural network, trained on a large/general audio dataset. Some starting points could be PANNs, YAMNet or OpenL3.
Your overall approach is valid, but a major challenge in your case would be to avoid overfitting on the single test set and contaminating your results. I would recommend two things: 1) split the test set recording into a validation set, used for hyperparameter tuning 2) construct another dataset to form a similar task. And use that for all early experimentation.
