Distribution of classes in neural network batches I'm creating a neural network for classifying input data. When using batches, do I need to ensure a somewhat uniform distribution of classes per batch? Or can I simply split up my data in any which way?
 A: Uniform distribution of classes per batch is best, since it's a better approximation of the gradient computed on the entire training set. In practice, batches are typically chosen randomly.
[1] addresses the question:

As for any stochastic gradient descent method (including
  the mini-batch case), it is important for efficiency of the estimator that each example or minibatch
  be sampled approximately independently. Because
  random access to memory (or even worse, to
  disk) is expensive, a good approximation, called incremental
  gradient (Bertsekas, 2010), is to visit the
  examples (or mini-batches) in a fixed order corresponding
  to their order in memory or disk (repeating
  the examples in the same order on a second epoch, if
  we are not in the pure online case where each example
  is visited only once). In this context, it is safer if
  the examples or mini-batches are first put in a random
  order (to make sure this is the case, it could
  be useful to first shuffle the examples). Faster convergence
  has been observed if the order in which the
  mini-batches are visited is changed for each epoch,
  which can be reasonably efficient if the training set
  holds in computer memory.


[1] Bengio, Yoshua. "Practical recommendations for gradient-based training of deep architectures." Neural Networks: Tricks of the Trade. Springer Berlin Heidelberg, 2012. 437-478.
