0
$\begingroup$

I am working on an assignment which is set to recognize on of 6 basic human emotions based on facial expression data.

The data set looks like this:

  • input data: Nx136, where N is the total number of examples and 136 is the dimensionality of the feature vector computed by concatenating the coordinates of 68 facial points.
  • target data: Nx1, containing the emotion labels. These labels are labeled from 1 to 6.

I would like to perform 10 fold cross-validation evaluation of my neural network using the parameters and an optimal learning rule.

I understand that I need to partition my data into 10 non overlapping folds. In my case there are 448 samples to work with. Each sample corresponds to a target label (based on the column number). I assume that the folds has to be almost the same size, so I'll have 8 folds with 45 samples and 2 with 44 samples. The target labels are ordered (first there are all the label 1s, then the label 2s and so on), hence the corresponding facial coordinate samples are as well. There different number of samples for each facial expression, in my case [47 60 68 113 58 102].

Shall I aim to have an equal number of the mixture of each label group in my folds? e.g. roughly 8 of each

I am stuck with implementing a generic function that would ensure an equal distribution of samples for each target label per partition. I am finding it difficult as the number of samples in the folds are not the same (there two with 44 samples the rest with 45) and also there are different number of samples for each label.

Could anyone suggest a logical way of doing this so that it could be apply for any k-fold partitioning in MATLAB?

Improve this question
$\endgroup$
2
  • $\begingroup$ There is no requirement to do 10 fold CV. Since you only have 448 samples you can do 2 fold, 3 fold etc. Depending on how long your ANN takes to train you can do leave-out-out CV to avoid any decisions on how to partition your data samples also. $\endgroup$
    – bill_e
    Commented Feb 19, 2015 at 15:54
  • $\begingroup$ I am actually required to do a 10 fold for this particular exercise. $\endgroup$
    – jaydb
    Commented Feb 19, 2015 at 16:05

1 Answer 1

Reset to default
1
$\begingroup$

You can use Optunity's cross-validation facilities to generate the folds. Whether or not you want to stratify (i.e. distribute instances of each class equally over folds) depends on you (the software allows it).

With unequal numbers of instances per class you can't expect to get a fully equalized coverage in each fold; you'll probably end up with some folds that have one extra instance for a given class compared to others..

Improve this answer
$\endgroup$
1
  • $\begingroup$ Is there a way to do easily do it in MATLAB? $\endgroup$
    – jaydb
    Commented Feb 20, 2015 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.