If initialised randomly and fed the same input, what stops convnet layer features from becoming the same? Re-posting this from scicomp based on recommendation in the comments.
(I'm a total ML noob, apologies if I've worded things badly, used the wrong terms or posted in the wrong SE site!)
I've been watching Magnus Erik Hvass Pederson's Tensorflow tutorials on YouTube.
One of the things I was previously confused about was where the set of features a convnet detects comes from, however after watching these videos I now believe these are learned; starting off by being initialised randomly.
In Tutorial #13 he shows a way of visualising what features are being detected by each layer (eg.: 9:36):

My question is; if these things are all initialised randomly and fed the same input data, what's to stop these features ending up the same? (Or another way, what ensures they end up reasonably distributed to detect all the different things required?)
 A: 
however after watching these videos I now believe these are learned; starting off by being initialised randomly.

correct

If these things are all initialised randomly and fed the same input data, what's to stop these features ending up the same? 

Since the weights are initialized randomly, they'll receive different weight updates during backpropagation. If the weights were initialized to 0, then they'll all stay the same during the training phase, which we don't want.

So if the only difference between them is the initial random input, does this mean training with same params and the same training data twice might result in two completely different results (and that one might pick up great features and work really well and another pick up poor/duplicated features and perform terribly)? 

In theory, yes. In practice, for image classification, the learnt features tend to be quite similar. But  feature  1 of your neural network 1 could (loosely) correspond to feature 29 of your neural network 2. Since ensembling a set of trained neural networks often yield higher performances than a single trained neural network, this means that the learnt features aren't exactly the same.

Wouldn't we be better initialising with some known good starting points (shapes, patterns, etc.) instead of random noise?

I guess they must have been some papers published about it in computer vision, but I don't have any off the top of my head. In natural language processing, it is common to initialize word vectors with pre-trained word vectors. Sometimes it helps, sometimes not.
