It seems to me that ConvNet is only useful when applied to convolutional data like images and sound, where data has a continuous nature (data points that are near often have similar values or correlated).
The other deep network model I know is Hinton's stacked restricted Boltzmann machines, where each layer is like an autoencoder and you pre-train it layer by layer.
And in ConvNet the current state of art is that you don't pretrain, just use ReLU neurons and brute-force backprop will do. So the only innovation in training algorithm is the stacked RBMs.
After these two successes, are there any other noteworthy models that has developed new training algorithms or new structures? (I know there are models like neural turing machines and long-short term memory but they look very artificial and there's no reason to expect they can do well, unlike ConvNet where convolution and pooling can be somehow explained so it feels natural.)
For example when the data is NOT continuous, for example you have many indicators of a certain stock value, are there good deep NN models to use in this case?