-1
$\begingroup$

Every epoch I'm logging the accuracy of a deep learning method on the test set. These are the results so far. The whole run is 150 epochs

0,33.6057
1,38.0719
2,62.037
3,66.3943
4,64.6514
5,44.4989
6,44.2266
7,59.5861
8,45.7516
9,47.7124
10,49.6732
11,48.5839
12,45.8606
13,50.1089
14,52.3965
15,56.7538
16,53.3769
17,51.6885
18,57.7342
19,58.6601
20,57.0806
21,59.3682
22,41.7756
23,42.1024
24,58.9869
25,57.5163
26,57.4619
27,58.4423

The value after the comma is the accuracy percentage.

Is this a normal behavior or are the changes in accuracy too big and inconsistent?

What general things can I further explore to improve on the results?

Edit to add more information:

It is a semi-supervised model

Labeled data size for training: 7408

Unlabeled data size for training: 46077

Test data size: 1836

Mini-batch size: 100

Layers: [3000, 4000, 2500, 1000, 500, 250, 2]

Activation: lrelu for hidden layers and softmax output layer

Learning rate: Testing both 0.01 and 0.02. Both behave similarly as far as I can see. Decay after 15th epoch

It is an implementation of this model with some few changes: https://github.com/rinuboney/ladder

EDIT 2:

Actually learning rate 0.01 seems to be behaving more estable

$\endgroup$
  • $\begingroup$ To help answer this, please add some details about your data set size, and the train/test split. Also, what the training set accuracy is - perhaps you are over-fitting after epoch 4? Specific to deep learning might be things like mini-batch size and what you are using for regularisation. If this is a problem where you have a benchmark from other learners (maybe including human experts) that might be of interest to gauge whether there is a real problem. $\endgroup$ – Neil Slater Sep 3 '17 at 15:40
  • $\begingroup$ @NeilSlater I added some additional info $\endgroup$ – Atirag Sep 3 '17 at 15:50
  • $\begingroup$ To follow Neil Slater, what are the results with few hidden layers (to check overfitting). Also, which quantity is optimized at each step of the algorithm (and what are results on training and test sets at each step for this quantity) ? Graphical representation can also help $\endgroup$ – ahstat Sep 3 '17 at 15:59
  • $\begingroup$ @ahstat ok it will take me a while to test this because the model is kind of heavy so I run it remotely on a grid. Any suggestions how to visualize the data remotely? I have no idea about how to do it but will try it if there's a simple way $\endgroup$ – Atirag Sep 3 '17 at 16:14
0
$\begingroup$

How many classes do you have, two? In this case 50 percent accuracy is random and your model learned nothing so far.

Is your data unbalanced? (Like 95 percent black Pixel and 5 percent White Pixel) . In this case you should Check precision, Recall and F1 Score.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.