Choosing optimal Batch Size : contradicting results? I'm a grad student in Mathematics, with little background in Machine Learning. I've recently come across the paper "A Disciplined Approach to Neural Network Hyper-Parameters : Part 1" by Leslie Smith, and I am really confused about his approach in Batch Size. 
He proposes that when using the "1-Cycle Policy" to a model one should use larger batch sizes, contrary to earlier works saying that small batch sizes are preferable. (My small experimental models seems to support his idea  too.) Many papers argues that using large batch sizes might be computationally inexpensive, but will lead to poor generalization, due to the fact that large batch size tends to converge to a sharp optimal point for unknown reasons. 
In conclusion, I should use large batch sizes ONLY when I'm using CLR, and smaller batch sizes otherwise, right? But why? These two facts seems to contradict one another, and maybe it's because of my major or something, and it's hard for me to just "accept" this fact purely on the test results. I would be happy if someone could give me even a vague reason behind this!
Here's a link to the paper I'm referring : A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay
[Edit] Here's a quote from "On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima" (Nitish Shirish Keskar, et al.)

One natural avenue for improving parallelism is to increase the batch size |Bk|. This increases the amount
  of computation per iteration, which can be effectively distributed. However, practitioners have observed that this leads to a loss in generalization performance; see e.g. (LeCun et al., 2012). In other
  words, the performance of the model on testing data sets is often worse when trained with largebatch methods as compared to small-batch methods. In our experiments, we have found the drop in
  generalization (also called generalization gap) to be as high as 5% even for smaller networks.

and here's the link : On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
 A: Indeed, there are many claims that large batch sizes do just as well, and just as many claims on the other side. Basically, it doesn't look like there's any consensus on the matter yet. For either side you can find as much theoretical justification as you'd like. (Obviously these aren't "actual" mathematical contradictions, since different authors start from different assumptions.)

These two facts seems to contradict one another, and maybe it's because of my major or something, and it's hard for me to just "accept" this fact purely on the test results.

I like to have theoretical justification as much as anyone, but neural networks is one place where the theory hasn't really caught up, and when the empirical evidence contradicts what theory suggests, it's probably a better idea to go with the evidence. 

Priya Goyal et al. "Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour" https://arxiv.org/abs/1706.02677
Elad Hoffer et al. "Train longer, generalize better: closing the generalization gap in large batch training of neural networks" https://arxiv.org/abs/1705.08741
Christopher J. Shallue et al. "Measuring the Effects of Data Parallelism on Neural Network Training." https://arxiv.org/abs/1811.03600
Nitish Shirish Keskar et al. "On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima" https://arxiv.org/abs/1609.04836
Tao Lin et al. "Don't Use Large Mini-Batches, Use Local SGD" https://arxiv.org/abs/1808.07217
Dominic Masters and Carlo Luschi. "Revisiting Small Batch Training for Deep Neural Networks." https://arxiv.org/abs/1804.07612
