0
$\begingroup$

I am working on time series multiclass classification task.

My data consists of 5 features (3 categorical and 2 numerical features) and 3 classes target value. Here are the histograms of the class distribution for train dataset:

y_train hist

Here is the summary of my network

Model: "model_3"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
day_of_week_input (InputLayer)  [(None, 3)]          0                                            
__________________________________________________________________________________________________
day_of_month_input (InputLayer) [(None, 3)]          0                                            
__________________________________________________________________________________________________
month_input (InputLayer)        [(None, 3)]          0                                            
__________________________________________________________________________________________________
day_of_week_emb (Embedding)     (None, 3, 7)         49          day_of_week_input[0][0]          
__________________________________________________________________________________________________
day_of_month_emb (Embedding)    (None, 3, 31)        961         day_of_month_input[0][0]         
__________________________________________________________________________________________________
month_emb (Embedding)           (None, 3, 12)        144         month_input[0][0]                
__________________________________________________________________________________________________
numeric_input (InputLayer)      [(None, 3, 2)]       0                                            
__________________________________________________________________________________________________
concatenate_6 (Concatenate)     (None, 3, 50)        0           day_of_week_emb[0][0]            
                                                                 day_of_month_emb[0][0]           
                                                                 month_emb[0][0]                  
__________________________________________________________________________________________________
concatenate_7 (Concatenate)     (None, 3, 52)        0           numeric_input[0][0]              
                                                                 concatenate_6[0][0]              
__________________________________________________________________________________________________
dense_9 (Dense)                 (None, 3, 32)        1696        concatenate_7[0][0]              
__________________________________________________________________________________________________
spatial_dropout1d_3 (SpatialDro (None, 3, 32)        0           dense_9[0][0]                    
__________________________________________________________________________________________________
lstm_3 (LSTM)                   (None, 16)           3136        spatial_dropout1d_3[0][0]        
__________________________________________________________________________________________________
dropout_3 (Dropout)             (None, 16)           0           lstm_3[0][0]                     
__________________________________________________________________________________________________
dense_10 (Dense)                (None, 8)            136         dropout_3[0][0]                  
__________________________________________________________________________________________________
dense_11 (Dense)                (None, 3)            27          dense_10[0][0]                   
==================================================================================================
Total params: 6,149
Trainable params: 6,149
Non-trainable params: 0
__________________________________________________________________________________________________

Here is how I train the model

METRICS = [
      keras.metrics.CategoricalAccuracy(name='accuracy'),
      keras.metrics.AUC(name='auc'),
      keras.metrics.Precision(name='precision'),
      keras.metrics.Recall(name='recall')
]

model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=[METRICS])

history = model.fit({'numeric_input': X_tr_numerical.astype('float32'),
                     'day_of_week_input': X_tr_cat1.astype('int32'),
                     'day_of_month_input': X_tr_cat2.astype('int32'),
                     'month_input': X_tr_cat3.astype('int32')}, y_tr,\
                    epochs=500, batch_size=64,\
                    validation_split=0.2,
                    verbose=2, shuffle=True)

The train/validation loss plot is the following Loss plot

My problem is that after 500 epochs classification both train and validation metrics are bad: train accuracy: 0.5018, train auc: 0.6732, train precision: 0.5931, train recall: 0.2473.

val_accuracy: 0.4748, val_auc: 0.6210, val_precision: 0.5476, val_recall: 0.1655

Could you please give me a piece of advice on how to improve the classification metrics?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.