Possible to get a better ANN by removing some connections? I was wondering if there under some circumstances is possible for ANN's to perform better if you prune away some connections on them as for example:
Constructing one ANN by taking two multi-layered ANN's A and B in parallel (same input and output nodes) adding a few "communication" connections between the hidden layers of A and B?
Could one get better generalization results?
Is this somehow used in practice or does one always just use multi-layered fully connected networks?
 A: Yes it is possible. Some people have looked at this problem in detail. Here is an old paper about a method to do so: Optimal brain damage
A: As a rule of thumb, small and/or sparse networks generalise better. You can let your training algorithm weed out unecessary connections within a fixed-size network by applying some form of weight decay, or you can apply an algorithm that aims to optimise network architecture/topology itself through removing unecessary inputs, hidden nodes or connections.
Have a look at these references for ideas and starting points for further research, or look into the use of evolutionary algorithms to design, prune and optimise architectures.


*

*Castellano, G., Fanelli, A.M. (2000) 'Variable selection using
neural-network models', Neurcomputing (31)

*Ji C., Psaltis D. (1997) 'Network Synthesis through Data-Driven
Growth and Decay', Neural Networks Vol. 10, No. 6, pp. 1133-1141

*Narasimha P.L. et al (2008) 'An integrated growing-pruning method
for feedforward network training', Neurocomputing (71), pp.
2831-2847

*Schuster, A. (2008) 'Robust Artiﬁcial Neural Network Architectures',
International Journal of Computational Intelligence (4:2), pp.
98-104

A: In most cases if you remove unnecesary connections you'll get better network. It is easy to overtrain (overfit) the network --- in which case it will perform poorly on validation dataset. 
Pruning unnecesary connections will most probably reduce o overtraining probability. Please see: http://en.wikipedia.org/wiki/Overfitting .
A: Yes It is possible.
We can consider, connection between computational unites, number of hidden layers, unites per hidden layer etc as hyper-parameters. It possible to find-out optimal values for these parameters by conducting a series of experiments.
For example:
Your can divide your data set as follows:
Training set 60% of data,
Cross-validation 20% of data,
Testing 20% of data,
Then train your NN by using training data set and tuning parameter by using cross-validation data set.
Finally you can use your testing data set for evaluate the performance of your NN.
