LSTM input and output parameter (not time series, not NLP)

Question

I have asked this question on stackoverflow but nobody answers. So I come here in the hope that somebody could help me solve it. Thank you! Here is my question:

I am a little bit confusing about how should I set parameters in my Keras LSTM model. The dataset is like this:

 print(x_train.shape, y_train.shape)
 (1000, 626) (1000, 225) #This is a multilabel dataset. Each label has two possibilities either 0 or 1.

I checked Keras document, it says like this:

hidden_units, time steps, input dimension 

So should I set it as:

model.add(LSTM(40, activation='relu', input_shape=(626, 1)))

or

 model.add(LSTM(40, activation='relu', input_shape=(1,626)))

Note, the hidden units is calculated=40, but then it's smaller than both input and output dimensions. See the following tables:

Tabel one: set input as hidden_units=40, input_shape=(1,626)

 Model: "sequential_1"
 _________________________________________________________________
 Layer (type)                 Output Shape              Param #   
 =================================================================
 LSTM_1 (LSTM)                (None, 1, 40)             106720    
 _________________________________________________________________
 LSTM_2 (LSTM)                (None, 1, 20)             4880      
 _________________________________________________________________
 Dense_1 (Dense)              (None, 1, 250)            5250      
 _________________________________________________________________
 Dense_2 (Dense)              (None, 1, 225)            51706     
 =================================================================
 Total params: 168,556
 Trainable params: 168,556
 Non-trainable params: 0
 _________________________________________________________________

Tabel two: Set hidden_units=40, input_shape=(626,1)

  Model: "sequential_2"
  _________________________________________________________________
  Layer (type)                 Output Shape              Param #   
  =================================================================
  LSTM_1 (LSTM)                (None, 626, 40)           6720      
  _________________________________________________________________
  LSTM_2 (LSTM)                (None, 626, 20)           4880      
  _________________________________________________________________
  Dense_1 (Dense)              (None, 626, 250)          5250      
  _________________________________________________________________
  Dense_2 (Dense)              (None, 626, 225)          56475     
  =================================================================
  Total params: 73,325
  Trainable params: 73,325
  Non-trainable params: 0
  _________________________________________________________________

In both bases, the output is set based on the dimension of output shape. I 've seen on stackoverflow people have different answers. e.g. this and this

So in this case what is the right way to set those parameters (hidden_units, input_shapes) and output_shapes

Thanks of your help!

Answer 1

According to Keras documentation, the expected input_shape is in [batch, timesteps, feature] form (by default). So, assuming 626 features you have are the lagged values of a single feature, the input shape should be of size (None,626,1), where the first None represents the batch size. If you pay attention to number of parameters for the first case, i.e. input_shape = (1,626), the first LSTM layer has significantly higher number of params compared to the second case. This is because the last dimension value, 626, represents the number of features not the time steps, as if the input is a multivariate time series and for each of them you need a weight, which makes the number of parameters go high.