Using neural network trading in stock exchange, part 2

Question

In several months, I have developed a framework for using neural networks (FANN library) in a chart trading software.

The framework allows me to combine any inputs for the NN, choose a learning rate, momentum, activation function's variables, number of layers and number of neurons in them...almost anything FANN library offers. The following is the UI.

(tl;dr version below)

Basically, I don't know what to do with it now.

The internet has been so far my only source of information and I'd appreciate ANYTHING since I'm pretty much stuck right now and can't move anywhere (the project is a requirement for my master's degree).

I have tried different setups and right now, I'm just aimlessly changing the inputs and the parameters of the NN. The problem is, everything I do and then start with teaching, the NN is just approximating itself to zero.

I have tried incremental learning, batch learning, various values of learning rate and momentum. As an output with which I'm teaching the network, I'm using the slope of the linear regression curve for the following X bars (10 in the picture in the field Output Window).

I've been using all sorts of things to the input. The picture shows two CCI oscilators - they are both having 10 as the input window which means there are total 20 inputs to the neural network, the difference between them is the length of the inner simple moving average (15 and 50) and they are both scaled by the maximum and minimum of the whole data set to the range -1, 1 since I'm using the bipolar sigmoid as the activation function.

But really, I can insert the price itself, the volume or the day/hour, any average or any derivative such as the CCI as the input and choose how it should be scaled (by the min/max of the whole dataset or by the min/max of the input window). I have tried simple things like choosing only the close prices of the last 10 bars or incredibly complicated things like three CCIs with various lengths, two exponential moving averages, hour, day, last volume change.....there were over 50 inputs, I've used different hidden layer setups...

...but in the end, the neural network is just approximating to zero.

I don't know what that means. I thought that the neural network is simply saying "Don't bother with that, there is no relation between the inputs you use and the linear regression curve slope." but then I think it cannot be true - I can see there really are double tops and double bottoms, I can see that there really are some patterns combining the two CCIs that are working repeatedly (patterns described in a book for intraday trading begginers).

I'm using 3 months of trading data of the S&P 500 e-mini grouped by 1000 ticks. I was thinking that maybe 3 months is too much, I should probably try just one month and, well, the approximation to zero is still occuring but it is only taking a longer time. Only by using a really limited dataset (like just a few days), the neural network is actually trying to fit itself but I can clearly tell that it is being overtrained for such a short timespan.

I have been studying the neural networks a lot, yet I feel I have made some false assumptions about how they should or should not be used. So far, I thought that using various inputs and the linear regression slope as the output will lead the NN to find a relation between the two. I'd like the NN to exaggerate even the slightiest relation found and I thought that might be done using the high momentum and high learning rate...NN is still approximating to zero :(

Maybe NN is not supposed to work like that at all ... I've learned a lot about them yet I know very little to come to any conclusions. If you have anything you think could help me (apart from leaving the project), I'd be very grateful.

TL;DR:

My neural network is approximating to zero for different sorts of inputs with diferrent settings, do you know why?

Also, any ideas, advices, interesting sources on the topic of neural network usage for trading? Or do you know, at least, where else to ask?

EDIT - problem found, see my answer below

Answer 1

In my experience, when the neural network ends up with all zeros at the output, it is likely that the rails are being saturated. That is that the common sigmoid function processing element will only swing to 0 or +1 limits and if the input happens to be far outside of the linear input sensitivity threshold range, it will force the sigmoid to stick at one of the output levels. You could check if all of the inputs are limited to a small range (say -1 to 1) or force them to be limited by using a transformation, such as a scaling or normalization of data. And do not use raw price, but an approximately stationary transformation such as log returns.

There is a simple tutorial on this blog, but keep in mind it is using Weka, a java based GUI.

Answer 2

Thanks to pat, I re-checked the scaled values of my output and found out that scaling of my output is wrong.

The original output was in a range from -1 to +3, my original idea was to scale that to -1 to +1 lineary but now I see that scaling that logarithmicaly would make more sense, since the linear regression slope most of the time oscilates somewhere within the range -0.5, +0.5. With scaling by the min&max extremes, I pushed the most common values to the range -0.03 to +0.03 (approximately).

Answer 3

This is a question after my own heart as I am in the middle of developing a neural net based trading system myself, and in the past I have looked at using FANN in GNU Octave via its octave bindings.

Since, as you say in your question,

Basically, I don't know what to do with it now

I think the archives of this blog, which I've only recently come across myself, might help. There are several entries dating back a few years where the blogger talks about some of the issues involved in developing a neural net based trading system. Another useful link might be this one, which links to various academic papers which discuss the use of neural nets in trading. Finally, as an unabashed plug, you might find my blog a source of some inspiration.

Answer 4

I wouldn't suggest using regression slope as an output target, but rather another metric that's lagged into the future by either 2 days or 10-days. Two key issues with data inputs for a neural network is a), most people input all the data (bars) available, and b) input a lot of highly correlated predictors. If correlated features are input into a NN, the net will spend precious time learning the correlation -- which is undesirable. In addition, many bars will be non-informative and redundant, which for example, means that you don't need to input all the e.g. daily bar data if you are doing end of day (OED) trading. Although, I have no conflict of interests, look at the WAV and DDR modules developed by Jurik Research, which are available for most packages. WAV significantly reduces the bars needed for input, and DDR decorrelates input predictors you think are important.