I'm curious if there are any neural network packages out there that easily allow one to construct feed forward neural networks with shared weights, but also allow for the training to be done in parallel. Torch7 allows for easy construction of shared weights, although the parallel training support is either not there (or not documented well enough to make it obvious that it is). If it interfaced to Python, that would be even better, but this is not a requirement.
It sounds like Pylearn2 may do what you want. It has two implementations for convolutional networks which requires some amount of weight sharing. Furthermore one of these implementations is optimized for use on a GPU via CUDA using Theano. See the documentation for more information. I believe the associated code is pylearn2.models.maxout.MaxoutConvC01B.
I can't add a comment to @Emre's answer because I don't have enough points. You can train shared-weight networks in torch, be that using CUDA or not. The weight-sharing is supported for any tensor type. Training is done in parallel when you wrap the two shared modules in a nn.Parallel container
We use this in torch quite a lot, to build siamese networks.