In my opinion correlated input data must lead to overfitting in neural networks because the network learns the correlation e.g. noise in the data.
Is this correct?
The question as such is a bit general, and mixes two things that are not really related. Overfitting usually is meant as the opposing quality to being a generalized description; in the sense that an overfitted (or overtrained) network will have less generalization power. This quality is primarily determined by the network architecture, the training and the validation procedure. The data and its properties only enter as "something that the training procedure happens on". This is more or less "text book knowledge"; you could try "An Introduction to Statistical Learning" by James, Witten, Hastie and Tibshirani. Or "Pattern Recognition" by Bishop (my favourite book ever on the general topic). Or "Pattern Recognition and Machine Learning", also by Bishop.
For the correlation itself: Consider the input space having a certain dimension. No matter what transformation you use, the dimensionality will remain the same -- linear algebra says so. In one case the given base will be completely uncorrelated -- this is what you get, when you de-correlate the variables, or simply apply PAT (Principle Axis Transformation.) Take any linear algebra book for this.
Since a neural network with an appropriate architecture can model any (!) function, you can safely assume, that it also could first model the PAT and then do whatever it also should do -- e.g. classification, regression, etc.
You could also consider the correlation a feature, which should be part of the neural network description, since it's a property of the data. The nature of the correlation is not really important, unless it is something that should not be a part of the data. This would actually be a different topic -- you should model or quantify something like noise in the input and account for it.
So, in summary no. Correlated data means you should work harder to make the handling of data technically simpler and more effective. Overfitting can occur, but in won't happen because there is correlated data.
cherub is correct in regards to his statement's pertaining to over-fitting. However, I think the discussion of highly correlated features and ANN overly simplifies the issue.
Yes, it is true in theory that an ANN can approximate any function. However, in practice is it not a good idea to include numerous highly correlated features. Doing so will introduce many redundancies within the model. The inclusion of such redundancies will introduce unnecessary complexities and in doing so could increase the number of local minima. Given that the loss function of an ANN is not inherently smooth, introducing unnecessary roughness is not a great idea.