I am just starting in machine learning and wanted to make sure I understand the relationship between batch size and the number of features in the input data. I am referring specifically to the last chunk of code on this getting started guide provided by Keras:
and the FAQ on stateful LSTM's:
where it said:
explicitly specify the batch size you are using, by passing a batch_size argument to the first layer in your model. E.g. batch_size=32 for a 32-samples batch of sequences of 10 timesteps with 16 features per timestep.
Now, both in the getting started guide and the FAQ, the batch size is twice the size of the feature set of the input data... I just want to confirm that this doubling is only used as a "for instance" and the batch size could have been three times or four times the feature set (or data dimension), but that the batch size must be a multiple of the feature set and not something where there were 5 features and you could choose to go with a batch size of 7 or 12, right?
from keras.models import Sequential from keras.layers import LSTM, Dense import numpy as np data_dim = 16 timesteps = 8 num_classes = 10 batch_size = 32 # Expected input batch shape: (batch_size, timesteps, data_dim) # Note that we have to provide the full batch_input_shape since the network is stateful. # the sample of index i in batch k is the follow-up for the sample i in batch k-1. model = Sequential() model.add(LSTM(32, return_sequences=True, stateful=True, batch_input_shape=(batch_size, timesteps, data_dim))) model.add(LSTM(32, return_sequences=True, stateful=True)) model.add(LSTM(32, stateful=True)) model.add(Dense(10, activation='softmax')) model.compile(loss='categorical_crossentropy', optimizer='rmsprop', metrics=['accuracy']) # Generate dummy training data x_train = np.random.random((batch_size * 10, timesteps, data_dim)) y_train = np.random.random((batch_size * 10, num_classes)) # Generate dummy validation data x_val = np.random.random((batch_size * 3, timesteps, data_dim)) y_val = np.random.random((batch_size * 3, num_classes)) model.fit(x_train, y_train, batch_size=batch_size, epochs=5, shuffle=False, validation_data=(x_val, y_val))
Also, that the number of units initialized in the LSTM layers only just so happens to be 32, the same size as the batch size and that this number really could be anything, such as 40 or 50, etc.