How do you go about choosing initial hyperparameters (layer size, # of hidden units in RNN and dense layers, etc.) when training RNNs and MLPs? How do you iteratively tweak these settings -- do you generally start with changing the number of layers, units, or optimizer?
The first part of your question is very closely related to this question (How to choose the number of hidden layers and nodes in a feedforward neural network?). However, keep in mind that mostly, you figure out the layers and unit amount through trial and error.
Once you have found a suitable network architecture, you get on with the optimizer - you don't want your network to overfit the data for example. This part is related to reduce over/under fitting, but also making training faster by initialising your weights correctly. A list of gradient optimization algorithms.
Ofcoures, you can always change the number of layers/units at any given time after that.