0
$\begingroup$

I am working with multivariate time series data with multiple examples to train LSTM on, and Y is either 0 or 1 binary classification. Currently, I am using pad sequence layer in combination with a masking layer to make every sample have the same number of time steps.

In order to make predictions, I have to pad test data to make it consistent with train data, and I have to feed a sequence with 3 time steps in current example for illustration purposes.

Is there a way to make VALID predictions by feeding only one time step after training on sequences instead of the entire sequence? For instance, can I just feed time step 3 with padded 1 and 2 instead of complete sequence consisting of 1 , 2, and 3 for predictions and get the same probability as I would from trained model on 1, 2, and 3 for time step 3.

# The input sequences are
trainX = np.array([
        [
            # Input features at timestep 1 with padding
            [0, 0, 0],                
            # Input features at timestep 2 with padding
            [6, 2, 1],
            # Input features at timestep 3
            [5, 2, 3]
        ],
        # Datapoint 2
        [
            # Features at timestep 1 with padding
            [0, 0, 0],
            # Features at timestep 2
            [9, 8, 9],
            # Features at timestep 3
            [7, 6, 1]
        ]
    ])

# The desired model outputs are as follows:
trainY = np.array([
        # Datapoint 1
        [
            # Target class at timestep 1
            [0],
            # Target class at timestep 2
            [0]
            # Target class at timestep 3
            [1] 
        ],
        # Datapoint 2
        [
            # Target class at timestep 1
            [0],
            # Target class at timestep 2
            [0]
            # Target class at time step 3
            [0]
        ]
    ])

timesteps = 3
# Create neural network architecture
model = Sequential()
model.add(Masking(mask_value=0., input_shape=(timesteps, trainX.shape[2]), name='Mask'))
model.add(LSTM(33, kernel_initializer ='uniform', return_sequences=True, batch_input_shape=(None, timesteps, trainX.shape[2]), 
               kernel_constraint=maxnorm(3), name='LSTM'))
model.add(Dropout(0.2))
model.add(LSTM(33, return_sequences=False, kernel_constraint=maxnorm(3), name='LSTM-2'))
model.add(Dropout(0.2))
model.add(Dense(timesteps, activation='sigmoid', name='Dense'))
# set optimization parameters'
sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
#compile model
model.compile(loss="mse", optimizer="sgd", metrics=["mse"])
fitness = model.fit(trainX, trainY, epochs=200, batch_size=20, validation_data=(tstX, tstY), verbose=1)
predY = model.predict(tstX)

I want to be able to predict by just feeding one time step as follows:

testX = np.array([
        [
            # Input features at timestep 1 with padding
            [0, 0, 0],                
            # Input features at timestep 2 with padding
            [0, 0, 0],
            # Input features at timestep 3
            [5, 2, 3]
        ]]

Every time I feed testX with different values in last time step, the probability comes out to be the same. It seems that probability is dependent on the number of time steps with unmasked real data as opposed to the data itself. If I duplicate last time steps as follows, the probability changes even when the data is the same. Do I need to mask test data? Why do I need to feed a sequence instead of one time step to get correct predictions?

testX = np.array([
        [
            # Input features at timestep 1 with padding
            [5, 2, 3],                
            # Input features at timestep 2 with padding
            [5, 2, 3],
            # Input features at timestep 3
            [5, 2, 3]
        ]]

Again, if I train on the following, can I make valid predictions in agreement with train data by feeding just the last step? If I get probabilities [[0,0,0.68] by feeding entire sequence, how can I get the same by feeding just time step 3 [5,2,3]

trainX = np.array([
        [
            # Input features at timestep 1 with padding
            [0, 0, 0],                
            # Input features at timestep 2 with padding
            [6, 2, 1],
            # Input features at timestep 3
            [5, 2, 3]
        ]]
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.