4
$\begingroup$

I'm considering using a neural network on financial time series but rather than train the network on actual data I am going to train on a model of the data which is perturbed by random noise. This being the case, I will potentially have unlimited amounts of model training data. However I don't want to generate a huge amount of data and then train the model as it might take a very long time to reach a solution, and I have no idea how much data to actually generate.

What I am thinking of doing is training on a small amount of model data (say 5000 examples) and get the values for the hidden nodes, record these values, and then repeat again, thereby building up a distribution of values for each node. These distributions could then be bootstrapped to get the mean value per node, and the whole process would stop once the change in the bootstrapped mean values falls below a given threshold.

Edit - the purpose of the network will be to classify/label the time series over the recent past as being in one of a finite number of states, e.g. trending up/down, moving sideways, in congestion etc. These states can be modelled using synthetic data with known labels, and then on real data the network's job will be to identify which state the real data most closely resembles. This will be used as input to a separate decision making process.

My question is - is there any reason why this would not be a valid approach to take?

$\endgroup$
  • 1
    $\begingroup$ I don't understand why you want to do this. Using a model instead of real data means that you have tremendous confidence that the model is correct. What is the classifier doing to categorize the time series? $\endgroup$ – Michael Chernick Jun 14 '12 at 19:31
  • $\begingroup$ @MichaelChernick I appreciate that more information would be useful and I have edited my question accordingly. $\endgroup$ – babelproofreader Jun 14 '12 at 19:53
  • 2
    $\begingroup$ Given that financial time series are so difficult to model, I don't think it's a good idea... $\endgroup$ – Lucas Reis Jun 14 '12 at 20:12
  • $\begingroup$ Instead of doing that why couldn't you have a single series to represent each of the categories and put some band around each one to determine where the real series would need to fall close to the model series so as to be classified in that category? You would need any training data (real or synthetic) and you would need a neural network classifier either. $\endgroup$ – Michael Chernick Jun 14 '12 at 20:42
  • $\begingroup$ @MichaelChernick Your suggestion is actually one of the approaches I will use, but this is only one feature. I have numerous other features that I also want to use in this classification process, hence the desire to experiment with a NN. $\endgroup$ – babelproofreader Jun 14 '12 at 21:13
1
$\begingroup$

You are not the first person who does something like this. Researches have done this for image recognition for about 15 to 20 years. An example for this is the MNIST data set of handwritten digits. The data set usually consists of 60,000 training examples. But this number is not sufficient to reach >99,5 % accuracy with multilayer perceptrons. So, people generate more training examples with distortions in each iteration of the optimization algorithm. The algorithm they usually use to train the neural networks is called stochastic gradient descent (or online learning in comparison to batch learning). There exist variants like "stochastic diagonal Levenberg-Marquardt" that require an approximation of the Hessian. The averaging of weights could produce a classifier that is really bad.

$\endgroup$
  • 1
    $\begingroup$ Distorting the training samples to generate new, synthetic ones does add valuable information (in this example, at least), but "adding noise", as the OP suggests, just adds noise, with no new information, and is unlikely to improve accuracy. $\endgroup$ – Vincent Zoonekynd Jul 1 '12 at 13:48
  • $\begingroup$ I'm not sure about that. Of course, the gain of information is not that big. But you never know when you haven't tried. :) $\endgroup$ – alfa Jul 1 '12 at 14:10
  • 2
    $\begingroup$ Here is a paper that proposes adding noise to input data and shows improved classification results due to the added noise: citeseerx.ist.psu.edu/viewdoc/… $\endgroup$ – rrenaud Jul 30 '12 at 21:45
  • $\begingroup$ What is the name for the technique of auto-generating training data? How successful is it compared to using only real data? How prone is it to overfitting? $\endgroup$ – pete Aug 13 '15 at 16:53
  • $\begingroup$ This is usually very specific for the application. I don't know of any common name. In computer vision it is usually called distortion but this a term which is most likely too generic for Google. :) You can read about that e.g. in the latest publications about the imagenet dataset. $\endgroup$ – alfa Aug 13 '15 at 17:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.