Detection of music note sequence in audio signal

Question

I have an audio signal which contains the combination of different western music notes(I know this combination in advance) and I want to identify the sequence of the music notes present in it. For this, I am following the below steps:

• Firstly, I am identifying all the dominant frequencies present in that signal using PredominantPitchMelodia algorithm of essentia library of python.
• The output of this algorithm provides me the vector array of dominant frequencies.
• Now, I want to distinguish between different music notes present in the signal with the help of this vector array of frequencies.
• This array of frequencies also contains noise from the environment, which I don’t want to remove.

The above figure shows the combination of 8 notes which I need to differentiate.

Following are the methods I tried:

• I tried to identify the sequence using k-means clustering method. But k-means clustering is not suitable for time series signal. The main drawback of using k-means is that when we do clustering, the noise frequency present in one note might lie in the frequency cluster of other note. So instead of considering it as noise, the algorithm considers it in the cluster of previous note.
• I am also thinking to implement moving average but again I am doubtful that it might alter my data and the result might not be that accurate.
• I also thought of creating an adaptive window which could shift in time and do clustering frame by frame. No idea how to implement this adaptive window and whether it is a good method or not. Again for adaptive window, I should know the window size in prior. And if I could guess the window size to form cluster for one note then I actually don’t need to cluster anymore because I already have the boundary points.
• To find the boundary points, maybe segmentation could be useful which is used to detect silence in audio signals but my data need not contain silence between two notes. And again segmentation might require training data(supervised learning)

Please help me to figure out the solution for this problem.

Answer 1

Of course clustering is the wrong tool. Because you probably don't want to discover new tones. Instead you probably want to map the measured frequencies to the predefined notes...

Why don't you first map every detected frequency to the nearest "good" frequency? And then do a simple filtering to reduce noise. If you want something really fancy, define transition probabilities based on known data and use the Viterbi algorithm to find the most likely states.

Answer 2

You have a sequence of pitch observations, and want to match this to a known sequence. This is a classic problem within many areas, such as gesture detection and speech recognition. There are two typical approaches: classification or sequence decoding. Classification means training a supervised machine learning model to detect the sequence. This is good if you have few things to detect (1-100), but infeasible for large number of classes (say every possible sequence of 8 semitones).

Viterbi decoding

The Viterbi algorithm allows to output the most likely sequence that matches a sequence of observations.

This paper on Hidden Markov Models and Viterbi Algorithm explains how it can be used to decode chord sequences.

The librosa library now contains a Viterbi module that could be used for this.