Can i expect better results?

Question

I am currently training a neural network that should be capable, of mapping a set of audio samples to a set of mfcc features. The mapping is done using a neural network, which i am currently training.

I normalized the data range of the input data,between -0.9 to 0.9 such that i could use an activation function such as tanh, as it ranges between those values. My neural network consist of 3 layers. the first two layers uses tanh as activation functions. and the last one uses ELU with $\alpha$=100. such that the output can range between -100 to $\infty$.

print "Model definition!"
model = Sequential()

#act = PReLU(init='normal', weights=None)
model.add(Dense(output_dim=400,input_dim=400, init="normal", activation=K.tanh))

#act1 = PReLU(init='normal', weights=None)
model.add(Dense(output_dim=400,input_dim=400, init="normal", activation=K.tanh))


act4=ELU(100)
model.add(Dense(output_dim=13, input_dim=400, init="normal"))
model.add(ELU(100))

I am using MSE as loss for optimizing for and RMSprop as a optimizer. Not sure i have specific reason for that? but could you suggest anthing better?

I am currently training my model using cv with a 90/10 split. ...

Or.. It trains on 90 % of the data, but within these 90 is 10% used for validation.

But i am getting these result.. result: 1 kfold

Which infact is quite horrible since the output data range is -100 to 100. So what can i do bettter?

Answer 1

You can try with many other methods such as:

• random forests (RF)
• support vector machines
• GAMs
• etc.

If you use R, you might consider reading this book http://appliedpredictivemodeling.com/user2014/, which comes with a great R package. Personally, I love RF, they work just as good as most of available methods$^1$ and are very easy to tune.

$^1$ For a comparison study on classification problems, see Fernández-Delgado et al. (2014) Do we Need Hundreds of Classifiers to Solve Real World Classification Problems? JMLR 15, pp. 3133−3181. PDF