3
$\begingroup$

I have time series consisting of 15 points of time, each containing around 15 values/features. Each time series is one sample, I have thousands of samples. Possible output is either 0 or 1, so binary classification.

Currently I feed in all 15x15 = 225 values = 225 inputs (per time series), without distinguishing or weighing them in time or any other aspect, in a normal backprop net (3 hidden layers, but less also works, Matlab Patternnet, scaled conjugate gradient) simultaneously and get best results (easily hits desired performance and gradient). Whenever I feed them in as vectors, e.g. 15 inputvectors with 15 values each, which would represent the time series way better, results get way worse. This is the first problem, I do not understand.

Another problem is, that with feeding them in just as 225 equal inputs, information about position in time series is not really contained. I hope I could use this information to increase performance of my neural net by reaching a higher level of abstraction and preventing overfitting.

When X is the input matrix for training and T the target Data, then it looks like this in case 1 (just put alls 225 values in equally):

X: 225x30000 double, T = 1x300000 double

In case 2 it would look like this:

X: 25x30000 cell with each cell containing 25x1 double, T= 1x30000 cell with each cell containing 1 double

Q1: Is case 2 the correct format as input? Q2: Should not it have the same result than case 1?

Why does that make such a big difference in performance?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Recurrent neural networks are the best suited for time-series analysis but they can be somewhat cumbersome to train in practice. You can try couple of layers of convolutions in time direction (does it make sense to do the other direction also? E.g. frequency dimension would). This should be quite a bit faster than a regular NN and would allow you to add a couple of layers increasing model complexity.

Improve this answer
$\endgroup$
2
  • $\begingroup$ How would I add these convolution layers to my pattern recognition net in matlab? I know how to add normal layers and set their connection. And why is it important whether I use 15x15 or 225 as an input? $\endgroup$
    – SwingNoob
    Mar 1, 2017 at 7:44
  • $\begingroup$ I don't have MATLAB experience with neural networks, so you should search if it is even possible. It shouldn't matter which data structure you use but you probably lose information e.g. MATLAB only reads the first column of 15x15 matrix. $\endgroup$
    – Joonatan Samuel
    Mar 1, 2017 at 11:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.