I would like to know how to deal with data that misses a label, but is likely to contain the label in a weakly supervised setting.
Weakly supervised background
Since labeling is a time consuming and noisy process (e.g. Should a timestamp of an event be at 5.4 or 5.5 seconds?), some people instead take a weakly labelled approach. This approach only provides a label for the whole recording. E.g. an audio file contains events x, y, z without any temporal information for each time stamp. A database containing weak labels is the YouTube-8M database.
The first 2 chapters of the paper Data-Efficient Weakly Supervised Learning for Low-Resource Audio Event Detection Using Deep Learning does a good job describing the issue.
I'm turning Mozilla's Common Voice dataset into a weakly supervised dataset. I do this by concatenating audio files (4+-3 seconds) into 10 second files. Then as output label for the whole 10 sec audio file I say it contains e.g. the accent labels
["eng_us", "eng_england", "eng_australia"]. If I want to recognize 5 English accents, I would e.g. encode this into
[1 0 1 1 0] (with the other 2 being e.g.
Due to user's not always specifying their information, such as "gender", "accent" and such, the data might miss the label
"eng_england", while it is clearly in the voice data. Unlike the YouTube-8M, where I assume that a video containing e.g. an explosion, but not labeled as such is rather small, in the common voice dataset it is a high amount. E.g.
- What is the best way to encode the labels for these concatenated audio files?
- If a value is
NaN, would it be better to take a soft labeling approach i.e.
[0.2 0.2 0.2 0.2 0.2]?
- Would a concatenated file with labels
[0 0 1 1 0]give a problem, while it should actually have been
[1 0 1 1 0], because person A didn't say he/she has an