Re-posting this from scicomp based on recommendation in the comments.
(I'm a total ML noob, apologies if I've worded things badly, used the wrong terms or posted in the wrong SE site!)
I've been watching Magnus Erik Hvass Pederson's Tensorflow tutorials on YouTube.
One of the things I was previously confused about was where the set of features a convnet detects comes from, however after watching these videos I now believe these are learned; starting off by being initialised randomly.
In Tutorial #13 he shows a way of visualising what features are being detected by each layer (eg.: 9:36):
My question is; if these things are all initialised randomly and fed the same input data, what's to stop these features ending up the same? (Or another way, what ensures they end up reasonably distributed to detect all the different things required?)