I am currently training a neural network that should be capable, of mapping a set of audio samples to a set of mfcc features. The mapping is done using a neural network, which i am currently training.
I normalized the data range of the input data,between -0.9 to 0.9 such that i could use an activation function such as tanh, as it ranges between those values. My neural network consist of 3 layers. the first two layers uses tanh as activation functions. and the last one uses ELU with $\alpha$=100. such that the output can range between -100 to $\infty$.
print "Model definition!" model = Sequential() #act = PReLU(init='normal', weights=None) model.add(Dense(output_dim=400,input_dim=400, init="normal", activation=K.tanh)) #act1 = PReLU(init='normal', weights=None) model.add(Dense(output_dim=400,input_dim=400, init="normal", activation=K.tanh)) act4=ELU(100) model.add(Dense(output_dim=13, input_dim=400, init="normal")) model.add(ELU(100))
I am using MSE as loss for optimizing for and RMSprop as a optimizer. Not sure i have specific reason for that? but could you suggest anthing better?
I am currently training my model using cv with a 90/10 split. ...
Or.. It trains on 90 % of the data, but within these 90 is 10% used for validation.
But i am getting these result.. result: 1 kfold
Which infact is quite horrible since the output data range is -100 to 100. So what can i do bettter?