Number of input neurons in a LSTM Autoencoder My training data of shape (2110, 5, 29). When building a LSTM Autoencoder can the number of LSTM cells in my first LSTM layer be more than dimensions of the original input (i.e. 29)?
Is it always the case that having more input neurons than features will lead to the network just copying the input value to the remaining neurons?
So do we prefer this:
num_observations = X.shape[0] # 2110
num_features = X.shape[2] # 29
time_steps = 5
input_shape = (time_steps, num_features)

# number of LSTM cells = 100
model = LSTM(100, return_sequences=True, input_shape=input_shape)
...

over this?
num_observations = X.shape[0] # 2110
num_features = X.shape[2] # 29
time_steps = 5
input_shape = (time_steps, num_features)

# number of LSTM cells = 25 
model = LSTM(25, return_sequences=True, input_shape=input_shape)
...

 A: In general, the number of features, or dimension of your hidden state, $H$ can be greater than or less than the number of dimensions of your input $D$.
For instance, if you're doing classification, setting $H > D$ allows your model to generate new, derived features from your input.
On the other hand, if you're building an autoencoder, then the model will, in theory, be able to copy the input if all layers between the input and the output have dimension greater than or equal to $D$. The result does depend on your choice of activation function, however. If you choose a ReLU activation but some of your inputs are less than zero, for example, then it may not be possible to reconstruct your input exactly, certainly not with $D=H$ and even possibly if $D > H$. Moreover, pathological behavior has been observed in feedforward networks with $D=1$ and $H \gg D$ (see, for example, section 1.1 of Trask et al. (2018)).
In computer vision, for compression or denoising autoencoders, people typically choose $H < D$ so that the autoencoder learns to represent the image using fewer features or with less noise.
I can't point you to specific examples, but I'm sure similar principles apply to other fields for tasks involving sequences.
