0
$\begingroup$

I am currently training a neural network that should be capable, of mapping a set of audio samples to a set of mfcc features. The mapping is done using a neural network, which i am currently training.

I normalized the data range of the input data,between -0.9 to 0.9 such that i could use an activation function such as tanh, as it ranges between those values. My neural network consist of 3 layers. the first two layers uses tanh as activation functions. and the last one uses ELU with $\alpha$=100. such that the output can range between -100 to $\infty$.

print "Model definition!"
model = Sequential()

#act = PReLU(init='normal', weights=None)
model.add(Dense(output_dim=400,input_dim=400, init="normal", activation=K.tanh))

#act1 = PReLU(init='normal', weights=None)
model.add(Dense(output_dim=400,input_dim=400, init="normal", activation=K.tanh))


act4=ELU(100)
model.add(Dense(output_dim=13, input_dim=400, init="normal"))
model.add(ELU(100))

I am using MSE as loss for optimizing for and RMSprop as a optimizer. Not sure i have specific reason for that? but could you suggest anthing better?

I am currently training my model using cv with a 90/10 split. ...

Or.. It trains on 90 % of the data, but within these 90 is 10% used for validation.

But i am getting these result.. result: 1 kfold

enter image description here

Which infact is quite horrible since the output data range is -100 to 100. So what can i do bettter?

$\endgroup$
1
$\begingroup$

You can try with many other methods such as:

  • random forests (RF)
  • support vector machines
  • GAMs
  • etc.

If you use R, you might consider reading this book http://appliedpredictivemodeling.com/user2014/, which comes with a great R package. Personally, I love RF, they work just as good as most of available methods$^1$ and are very easy to tune.

$^1$ For a comparison study on classification problems, see Fernández-Delgado et al. (2014) Do we Need Hundreds of Classifiers to Solve Real World Classification Problems? JMLR 15, pp. 3133−3181. PDF

$\endgroup$
  • $\begingroup$ why would using these method be better than a neural network?... $\endgroup$ – Bob Burt Nov 27 '16 at 14:54
  • $\begingroup$ @BobBurt I don't have a "proof" for that. The main point of the reference 1 I gave you in my answer is that, the authors compared 179 classifiers in 121 datasets and what comes out is that RF works best or it is comparable to SVM. Also NNets seem to do a good job, although not as good as RF. $\endgroup$ – utobi Nov 27 '16 at 15:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.