I'm having serious problems with this task. It already has weeks that I'm trying to achiev a way to train a non-squared (256x16, for example) data. How could I do that?
I'm trying to apply conv with ksize = [32,3] then max pooling it (I think it goes to [16,2] then...) but this is not working...
Please see this gist as a reference of the model(tensorflow):
https://gist.github.com/denisb411/ccb470ee31d66cdaf647264fb6e20576
Correct my commentary, if needed. I'm not sure if this is right...
Training CNN on non-squared data is ideologically the same. You can apply square-kernel convolution, max-pool it and whatever you normally do. The difference is that results after each layer has non-square dimension, which is not necessarily bad, but might become problematic in later layers, when one dimension will go down to 1 and another will still be high. My suggestion would be to use a larger stride in one dimension during convolution. For example in your case:
[256x16] -> conv([3,3], stride[2,1]) ->
[128,16] -> conv([3,3], stride[2,1]) -> [64,16]
...
Another suggestion: 1. convolutions with kernel size [32,3] is very very uncommon and I don't think it can work. Try to stick to smaller kernels (most commonly square). 2. stride of size [2,2] can sometimes replace max-pooling layer, you have to experiment.