Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
2 added 6 characters in body; edited tags
source | link

I'm working on the problem of automatically labeling groupings in music. In this domain, there's label at each time step indicating whether a group starts at this time step. However, labels are subjective, and not everyone agrees about the groupings.

I'm currently designing a study to collect these labels, and I'm trying to decide how many participants should listen to each sample. Of course this is a direct trade-off with the number of total samples that I get labels for. The number of samples will drive our choice of model, but we're particularly interested in RNNs.

How can I justify how many participants per sample I need? It could be as few as 1 person or as many as 10. 1 gives us more samples total, but 10 gives us better labels for each sample. How can I make a principled decision? My intuition says one person per sample, because that's what magenta and other rnn approaches to music tasks have done.

I'm working on the problem of automatically labeling groupings in music. In this domain, there's label at each time step indicating whether a group starts at this time step. However, labels are subjective, and not everyone agrees about the groupings.

I'm currently designing a study to collect these labels, and I'm trying to decide how many participants should listen to each sample. Of course this is a direct trade-off with the number of total samples that I get labels for. The number of samples will drive our choice of model, but we're particularly interested in RNNs.

How can I justify how many participants per sample I need? It could be as few as 1 person or as many as 10. 1 gives us more samples total, but 10 gives us better labels for each sample. How can I make a principled decision? My intuition says one person per sample, because that's what magenta and other rnn approaches to music tasks have done.

I'm working on the problem of automatically labeling groupings in music. In this domain, there's label at each time step indicating whether a group starts at this time step. However, labels are subjective, and not everyone agrees about the groupings.

I'm currently designing a study to collect these labels, and I'm trying to decide how many participants should listen to each sample. Of course this is a direct trade-off with the number of total samples that I get labels for. The number of samples will drive our choice of model, but we're particularly interested in RNNs.

How can I justify how many participants per sample I need? It could be as few as 1 person or as many as 10. 1 gives us more samples total, but 10 gives us better labels for each sample. How can I make a principled decision? My intuition says one person per sample, because that's what magenta and other rnn approaches to music tasks have done.

1
source | link

How does one trade off quantity of labels for accuracy of labels

I'm working on the problem of automatically labeling groupings in music. In this domain, there's label at each time step indicating whether a group starts at this time step. However, labels are subjective, and not everyone agrees about the groupings.

I'm currently designing a study to collect these labels, and I'm trying to decide how many participants should listen to each sample. Of course this is a direct trade-off with the number of total samples that I get labels for. The number of samples will drive our choice of model, but we're particularly interested in RNNs.

How can I justify how many participants per sample I need? It could be as few as 1 person or as many as 10. 1 gives us more samples total, but 10 gives us better labels for each sample. How can I make a principled decision? My intuition says one person per sample, because that's what magenta and other rnn approaches to music tasks have done.