I have a convolutional network (For details, please see the edit in the bottom) with training/testing errors that always look very similar to what is shown in the figure. In other terms, it seems that my model is overfitting without any generalization whatsoever from the begining. What I find also bizarre is its oscilliation, which is not present in the training error. I've tried enlarging the dataset, but the effects were almost negligible (the test error decreases for, say, 10 iterations, and then it starts oscilliating again.).
Am I wrong to think that a model that can overfit the training data is capable of modeling the problem? How should I interpret these errors?
Edit: The netwok architecture is given in the following figure. Basically, it takes two images as input. Net1 and Net2 are both simple convolutional architectures made of Convolutions, MaxPooling, and PRelu. I haven't used dropout, but I use batch normalization. The term SPP stands for spatial pyramidal pooling.