Dropout before Batch Normalization? In the last course of the Deep Learning Specialization on Coursera from Andrew Ng, you can see that he uses the following sequence of layers on the output of an LSTM layer:
Dropout -> BatchNorm -> Dropout.
To be honest, I do not see any sense in this. I don't think dropout should be used before batch normalization, depending on the implementation in Keras, which I am not completely familiar with, dropout either has no effect or has a bad effect. 
I might be missing something here, though, and if anyone has any knowledge of why something like this could be useful, I'd love to hear from them.
 A: The way I see it, it introduces much more noise into the model that a single batch normalization layer. But as shown in https://arxiv.org/pdf/1801.05134.pdf, dropout doesn't go well with batch normalization. Noone says Andrew Ng is infallible.
A: Your intuition that dropout changes the statistics is discussed in detail in this arxiv paper (to my knowledge, neither peer-reviewed, nor properly published anywhere else.)  I do not vouch for any of their proposed solutions. 
A similar paper that was submitted to ICLR 1017, narrowly rejected on the grounds of too incremental an improvement.  Likewise, I do not vouch for proposed solutions.
For what it's worth, my intuition is the same.  I tentatively advance the hypothesis that this is in the sour spot of, "mostly understood by the research community and so not often mentioned, analyzed, or written up; but also not quite obvious, especially to beginners or those with a kitchen sink mentality."  It surprising that Ng, who is not a beginner, would fall prey to that, but no one is perfect. 
