How does penalizing large weights (using the L2-norm) help prevent overfitting in neural networks? The effect of applying the L2-norm regularization in neural networks is that it penalizes large weights in the model. How does this prevent overfitting? 
My assumption is that large weights means that the neurons are heavily reliant on the output from other neurons. For example, in a multilayer perceptron, if one of my neuron has a large weight, it means that it relies heavily on a neuron in the previous layer. Using the L2-norm and forcing the weights to be smaller makes this neuron use the output from the previous layer in a more balanced fashion. Is this reasoning correct?
 A: This can be viewed in two ways. First, you can see L2 regularization as a prior on the weights to be distributed near 0. Since you have prior information on the possible distribution of weights, this rules out many improbable solutions. If your prior was correct, this will theoretically increase the chance that you get a reasonable answer -- one which will generalize and not overfit. This is sort of like Occam's razor -- in Occam's razor, we prefer short solutions to long solutions. Here, we prefer small weight solutions to large weight solutions.
The second way to see it is that a neural network is a function approximator which is extremely flexible and has a huge model capacity. By flexible I mean that it is relatively easy from an optimization standpoint to tweak the weights a little and arrive at your desired function. L2 regularization helps by lowering model capacity (too much of which causes overfitting) while leaving the network flexible enough to perform well even with the added regularization.
