1
$\begingroup$

I am working on classifying 30 second samples of music into one of four genres: ["electronic", "hip hop", "jazz", "rock"] and I would appreciate any kind of help.

I have generated my own data set from mp3 files. Within my "dataset" directory, I have arranged 100 songs by genre, putting 25 in subdirectories for each genre (i.e. I have "electronic", "hip hop", etc subdirectories).

So far, I have extracted 30 second samples of each of these mp3 files, normalized the signal so that no sample exceeds -32 or -18 dB, mixed them down to mono, and converted them to wav (with pyDub).

Next I used librosa to extract mfccs (mel frequency cepstral coefficients) for the 1292 frames in each song. Then I scaled the data so it has zero mean and unit variance using sklearn's preprocessing module.

I took these values and saved them to a csv file of MFFCs, where each row is a frame and each column is one of 12 coefficients.

So you can get an idea, the song followed by the first 20 frames: Julio Bashmore - Au Seve.mp3

1.6792870551627723,-0.3842983399875271,-0.4027844785642914,0.7165034707424635,-0.6823681099880697,0.8160136728323858,-1.5263184054951733,1.1145290823984928,-0.21784328023531235,0.047527570975473235,0.32866412875434237,1.869661743729989
-1.6792870551627723,-0.3842983399875271,-0.4027844785642914,0.7165034707424635,-0.6823681099880697,0.8160136728323858,-1.5263184054951733,1.1145290823984928,-0.21784328023531235,0.047527570975473235,0.32866412875434237,1.869661743729989
-1.6792870551627723,-0.3842983399875271,-0.4027844785642914,0.7165034707424635,-0.6823681099880697,0.8160136728323858,-1.5263184054951733,1.1145290823984928,-0.21784328023531235,0.047527570975473235,0.32866412875434237,1.869661743729989
-1.6792870551627723,-0.3842983399875271,-0.4027844785642914,0.7165034707424635,-0.6823681099880697,0.8160136728323858,-1.5263184054951733,1.1145290823984928,-0.21784328023531235,0.047527570975473235,0.32866412875434237,1.869661743729989
-1.6792870551627723,-0.3842983399875271,-0.4027844785642914,0.7165034707424635,-0.6823681099880697,0.8160136728323858,-1.5263184054951733,1.1145290823984928,-0.21784328023531235,0.047527570975473235,0.32866412875434237,1.869661743729989
-1.6792870551627723,-0.3842983399875271,-0.4027844785642914,0.7165034707424635,-0.6823681099880697,0.8160136728323858,-1.5263184054951733,1.1145290823984928,-0.21784328023531235,0.047527570975473235,0.32866412875434237,1.869661743729989
-1.6792870551627723,-0.3842983399875271,-0.4027844785642914,0.7165034707424635,-0.6823681099880697,0.8160136728323858,-1.5263184054951733,1.1145290823984928,-0.21784328023531235,0.047527570975473235,0.32866412875434237,1.869661743729989
-1.6792870551627723,-0.3842983399875271,-0.4027844785642914,0.7165034707424635,-0.6823681099880697,0.8160136728323858,-1.5263184054951733,1.1145290823984928,-0.21784328023531235,0.047527570975473235,0.32866412875434237,1.869661743729989
-1.6792870551627723,-0.3842983399875271,-0.4027844785642914,0.7165034707424635,-0.6823681099880697,0.8160136728323858,-1.5263184054951733,1.1145290823984928,-0.21784328023531235,0.047527570975473235,0.32866412875434237,1.869661743729989
-1.6792870551627723,-0.3842983399875271,-0.4027844785642914,0.7165034707424635,-0.6823681099880697,0.8160136728323858,-1.5263184054951733,1.1145290823984928,-0.21784328023531235,0.047527570975473235,0.32866412875434237,1.869661743729989
-1.6792870551627723,-0.3842983399875271,-0.4027844785642914,0.7165034707424635,-0.6823681099880697,0.8160136728323858,-1.5263184054951733,1.1145290823984928,-0.21784328023531235,0.047527570975473235,0.32866412875434237,1.869661743729989
2.4841536110972022,-0.5831476248573247,0.37058328670683277,-1.4599220579565508,-0.35449671920732007,-1.1326787825224918,0.5880762356956317,-0.8108172607843107,0.010134004741811507,-0.14931018884055094,0.8707111843072819,0.1667143116197902
2.0939135826765907,-0.4778720879089441,0.26530387765936375,-1.7076132053582773,0.11305806361775678,-1.310823349961563,1.1669240812438573,-0.8627333493359391,-0.19252158214293175,-0.039523355794829566,0.6658161856594883,0.2860711396454278
1.0127547898943148,-0.6547501371081066,-0.202002065081406,-1.7889468252345162,1.1632837017143651,-0.9288351063974712,2.070078331574107,-0.7601750354687623,-0.27909671541985936,-0.13713166210030908,0.2267359005199065,0.27808482310773774
0.06572087004052392,-1.8740496505118946,-0.9604185325425617,-1.0163364869696865,1.5840872642483552,-0.16659361108422382,1.7806813371087853,0.055159751832777354,0.6842054675590546,0.42350598071605017,-0.3324771084186967,-0.24348528197848257
0.7159690101152768,-2.6235135217332606,-0.9099658866643047,-0.19653348650619468,1.0348534863167884,-0.6771927176675163,1.0703663687805878,0.3981886714210787,0.8503521825769755,0.4055860454830591,-0.11841556456925736,0.05030541244676532
0.8810398765824345,-2.7727001749452045,-0.8484274387283207,0.14839104995756489,0.9124992899968386,-0.5987705973726993,0.6471053665081234,0.43190059553550836,0.9028748015921237,0.3425604687141461,-0.20209176692016032,0.15561852907964296
1.5217565976091,-2.5946551685896044,-0.3924558895014341,0.36743931340001096,0.9126773048246598,-0.7581004315396501,0.4463892360730688,0.42969123923287,0.7276796949470707,-0.0079165602005986,-0.580154306587985,-0.07235102966750707
2.08861621898524,-1.2804976691396324,0.46640912894919145,0.14007051920782673,0.9100754665932002,-1.51168507329552,0.7161640071116147,0.34780954351977644,0.30123647629161765,-1.103443008391695,-0.7900432022174468,-0.2847124076141728

I tried simply compiling the 1292 frame vectors for each song into one big vector for each genre, and use that as input for scikitlearn's kNN algorithm. I am getting very unfortunate results, I just get a vector filled with "rock":

['rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock', 'rock']

I'm fairly certain I'm not really going about it correctly at all, but I have the following two functions. The first one creates this feature vector for each genre. The second one simply trains it using each genre vector and a vector filled with labels for the genre.

def create_np_vector(db_read, start_row):
    num_frames = 1292
    num_songs = 25
    num_coefs = 12
    #vector for all features of every song/sample of that genre
    genre_vec = np.empty([(num_frames * num_songs), num_coefs])
    db_reader = csv.reader(db_read)
    for row in itertools.islice(db_reader, start_row, start_row + 25):
        id = row[0]
        path = row[2]
        mfcc_file = path + "/csv/" + id + ".csv"
        mfcc_reader = csv.reader(open(mfcc_file, 'r'))
        frame_num = 0
        for mfcc_row in mfcc_reader:
            frame_vec = np.array(mfcc_row)
            genre_vec[frame_num] = frame_vec
            frame_num = frame_num + 1
    return genre_vec

def train_knn(db_read, knn):
    genres = ["electronic", "hip hop", "jazz", "rock"]
    line_num = [0, 26, 51, 76]
    x = 0
    for genre in genres:
        vec = create_np_vector(db_read, line_num[x])
        x = x + 1
        print(genre)
        knn.fit(vec, [genre for x in range(25*1292)])

What exactly should I be doing here? I've been trying to use this as a resource: http://modelai.gettysburg.edu/2012/music/, but I'm still quite lost. Should I be calculating a mean vector and covariance matrix for each audio file?

Even if I did that, what would I do with the two vectors for each file?

$\endgroup$
0
$\begingroup$

So you have a poor result and you want to know what is going on?

  1. Poor features: Potentially your features are not allowing a good classification therefore you might consider new/more/other features. Yet to evaluate the features, you will have to compare the accuracy of other classifiers as well. (RandomForest, SVC)

  2. Poor classifier: In most cases (my experience) KNN yields worst results compared to other classifiers. It is the most simple classifier only works with proper distance measure on very simple data.

  3. Poor Training: You are not tweaking the parameters of your classifier. Even this most simple classifier KNN has a parameter which influences the prediction. You are using the default parameter, whereas you should use crossvalidation to determine the best possible parameter.

Finally I do not see where you are using a test set for evaluation? Where are you getting that "rock rock rock" array from? How do you split test and train set?

Note: Building a classifier that performs well is an iterative process and takes both time and patience. You will want to compare multiple classifiers and you will want to train them properly finding good parameters. Finally, if you are still not achieving good results, you will want to read papers on this topic, getting input of which other features might be relevant and helpful.

For you to continue working, start with 3 - train properly, then go to 2 - use other classifiers and only after all that work, go to 1- and compute new features.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.