1
$\begingroup$

I'm trying my hand at recurrent neural nets having successfully implemented a few simple feed-forward networks. I'm using tflearn and the following example code that performs a simple sentiment analysis using IMDB film reviews (see https://github.com/tflearn/tflearn/blob/master/examples/nlp/lstm.py):

from __future__ import division, print_function, absolute_import

import tflearn
from tflearn.data_utils import to_categorical, pad_sequences
from tflearn.datasets import imdb

# IMDB Dataset loading
train, test, _ = imdb.load_data(path='imdb.pkl', n_words=10000,
                                valid_portion=0.1)
trainX, trainY = train
testX, testY = test

# Data preprocessing
# Sequence padding
trainX = pad_sequences(trainX, maxlen=100, value=0.)
testX = pad_sequences(testX, maxlen=100, value=0.)
# Converting labels to binary vectors
trainY = to_categorical(trainY, nb_classes=2)
testY = to_categorical(testY, nb_classes=2)

# Network building
net = tflearn.input_data([None, 100])
net = tflearn.embedding(net, input_dim=10000, output_dim=128)
net = tflearn.lstm(net, 128, dropout=0.8)
net = tflearn.fully_connected(net, 2, activation='softmax')
net = tflearn.regression(net, optimizer='adam', learning_rate=0.001,
                         loss='categorical_crossentropy')

# Training
model = tflearn.DNN(net, tensorboard_verbose=0)
model.fit(trainX, trainY, validation_set=(testX, testY), show_metric=True,
          batch_size=32)

# Predict
model.predict(testX)

I noticed that if I call model.predict(testX) more than once, the output is not the same despite the input vectors being identical. Is this expected? If so, why?

Sorry in advance for the novice-level question

$\endgroup$
2
  • $\begingroup$ Are you simply repeatedly calling the model.predict() function after the net is fitted or rerun the whole python script? $\endgroup$
    – STATWORX
    Jul 9, 2016 at 22:39
  • $\begingroup$ Repeatedly calling the predict function, not re-running the entire script. In other words, I only call model.fit() once. $\endgroup$
    – maxp
    Jul 10, 2016 at 9:11

1 Answer 1

1
$\begingroup$

The reason is that currently there is a bug using dropout in RNNs, for some reason it still apply dropout at prediction time. We are checking that issue, it will hopefully be solved soon. So for now you can just remove dropout parameter from your lstm layer.

$\endgroup$
1
  • $\begingroup$ Thanks for the answer! I was starting to think I've gone crazy :) $\endgroup$
    – maxp
    Jul 10, 2016 at 19:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.