New answers tagged lstm
0
Here is the general recipe: http://karpathy.github.io/2019/04/25/recipe/,
You may specifically try to overfit one batch to check the model capacity.
activation to Relu -> Tanh, also suggested in the comment above by Alex.
Add another dense layer before the last layer to increase the capacity of the model to learn more nonlinearity of the curve Dense(4, ...
4
Yes, neural networks can learn how to play video games.
reinforcement-learning (RL) is the standard approach to solving game-playing using neural networks. A key paper in this area is Deepmind's Atari-playing RL agent, but researchers have extended this approach to more complex games like Doom, Starcraft II and DOTA. If you're not familiar with this ...
0
It's a bit too late but just in case;
A Sample may refer to individual training examples. A “batch_size” variable is hence the count of samples you sent to the neural network. That is, how many different examples you feed at once to the neural network.
TimeSteps are ticks of time. It is how long in time each of your samples is. For example, a sample can ...
0
This is an interesting question, but it is advanced topic that still hasn't been solved completely by researchers in the field. Some teams have tried to obtain representations of the embeddings after an LSTM is trained on time series data.
The only published paper I've seen so far is this one.
You might want too look at how people extract language ...
0
output[-1,:,:] will be the last slice of the tensor, i.e., the padding.
The real final states are in the second member of the tuple returned by lstm.
If you really want to get the state from output, you can use torch.gather.
torch.gather(output, dim=0, index=lengths - 1)
where lengths is a 1-D tensor with lengths of the sequences in the batch.
1
In the standard Transformer model as introduced by Vaswani et al. it is not possible, because generating a word is always conditioned on the previously decoded words, so there is no other option than generating words one by one.
Recently, there appeared several papers on so-called non-autoregressive models which parallelize the decoder as well, but the ...
0
That units in Keras is the dimension of the output space, which is equal to the length of the delay (time_step) the network is recurring to.
keras.layers.LSTM(units, activation='tanh', ....)
https://keras.io/layers/recurrent/
Top 50 recent answers are included
