I have a regression problem where I'd like to train an LSTM to tackle it. However, since I do not have too many samples for training (only 2000), I am thinking about using only one hidden node, since the representation of the input is a 300-d vector.
Does it make sense to use only one hidden node (I have never seen such architecture in papers)? Or should I just use something else (e.g. CRF) given the small amount of training samples that I have?