0
$\begingroup$

I am facing the following question, I want to train RNN with LSTM to classify data with temporal relations, the sting is that I want to classify every sample in the timeseries iteratively. I will give an example and then explain the problem I face.

An example is the following case: I want to classify whether a computer is infected with malware or not. For that purpose I sample it every minute and take the following feature vector: {cpu usage, %memory use, network load}. I want to create temporal pattern based on 128 samples back but obviously I can't wait for batches of 128 samples to be filled up and I need to classify the sample from the computer as I get a new one.

Now, my dilemma is around how to correctly feed the data to the NN and train using tensorflow. For a NN model which includes simple input->LSTM->class:

LSTM_cells = 200
num_classes = 2
lstm_cell = rnn.BasicLSTMCell(LSTM_cells, forget_bias=1.0)
state_in = lstm_cell.zero_state(1, tf.float32)

X = tf.placeholder(tf.float32, [None, 3])
Y = tf.placeholder(tf.float32, [None, num_classes])

X = tf.reshape(X, [1, -1, 3])
rnnex_t, rnn_state = tf.nn.dynamic_rnn( \
    inputs=X, cell=lstm_cell, dtype=tf.float32, initial_state=state_in)
rnnex = tf.reshape(rnnex_t, [-1, LSTM_cells])
out = tf.add(tf.matmul(rnnex, weights['out']), biases['out'])
logits = tf.reshape(out, [-1, num_classes])
prediction = tf.nn.softmax(logits)

# Define loss and optimizer
loss_op = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=Y))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)
train_op = optimizer.minimize(loss_op)

Specifically, I can either create a sample of length 1 every time and pass it to tf.nn.dynamic_rnn with sequence_length=1, in that case what should be the state? should it be updated? I can create an incremental array which will hold the last 128 and use the last classification, but again, should I reuse the state from last update? keep a different state for online classification and train? should I train the model on every iteration or wait for a full batch of 128 samples (completely disconnect the online and train)?

It feels like there are a lot of possible combinations and every one makes sense in a way but on the same time contradicts the "normal" way to use LSTM.

$\endgroup$
0
$\begingroup$

Although I think StackOverflow may better fit this kind of issues, I'd like to give some hints here. Hope it helps.

Since the batch size is unfixed you can just feed the sample one by one into the model in inference mode. You can define an iterative_session_run function to do that.

Since it is in the inference mode you should first train the model using lots of data, and save the variables/model after training and then load those variables in inference mode.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.