I am interested in researching a machine-learning algorithm for trading on Forex, after being inspired by several papers that I read. I want to do this more out of a love of computing and forex than anything. I understand the principles of input data, weights, learning, test/validation sets, etc, but despite the vast amount of information available on the net there are a few specific issues that are puzzling me, and I was hoping someone could point me in the right direction. I understand that Cross Validated isn't intended for 'open' questions but Im hoping my query doesn't fall into that category.

The situation is:

  • I have lots of 1-minute FX price/volume data available.
  • At any particular time, I (the program) have three options available which I'll assume are mutually exclusive - buy, sell, do nothing.
  • I dont know what the outcome of a decision is until the trade is closed, which is some time after the decision is made, and itself is open to optimisation/learning.

My problem is that many discussions on the internet are about "labelled" data, where the 'correct' result is already known. With trading, the outcome of the action isn't known until later, so how can I 'pair' the inputs and outputs into something that I can then calculate an error/cost from?

In the TensorFlow tutorial here [https://www.tensorflow.org/get_started/mnist/beginners][1] the author uses a Softmax function to link or map the inputs to an output. In his example though, he already knows that there is a strong correlation between his inputs and outputs and that a 'right answer' exists. He also seems to 'know' that a Softmax function is the right function to use to map his inputs to his outputs - aren't there many other models he could have used? In my situation I have no idea what the right type of model is, and I have no idea that a correlation even exists between my proposed 'inputs' and outputs?

If I was breaking the exercise I am about to undertake into high-level steps, I think I need to:

1) Decide what metrics I want to use. These are things that I suspect are important to making a trading decision - e.g. the bid-ask spread, the time of day or day of week, and inputs from indicators and technical analysis that I have prepared. I have no idea how many inputs are the right number of inputs. 2) Normalise the inputs so that they are treated fairly by the ML algorithm and turned into a single line of inputs as a 'vector' - e.g: v = [x1, x2, x3, m1, m2, m3, m4.... etc] Where for example x inputs are about spread information, m1-4 are inputs from my custom indicators, etc. I have no idea which of these are important and which ones aren't. 3) Select a batch of random datapoints V. Make TensorFlow (which Im proposing to use) run a number of iterations of the model. TensorFlow initially assigns random weights (the weights are my 'policy'?) to the inputs and determines its action for each input vector (v) in the dataset (V). The 'action' would be a vector in the form [y1, y2, y3] where they represent the "bias" of taking an action y1 (buy), y2 (do nothing) or y3(sell) and it will do whichever one has the biggest value, and y1 + y2 + y3 = 0 ? 4) For each input vector (v) that is given an action by the algorithm, I need to calculate its profit/loss, even though I don't know yet when the trade is closed. I also don't know how to 'break' TensorFlow to get it to pause whilst I run some code that works out what the Profit/Losses would be. Any pointers on this step in particular would be helpful! 5) Using the sum of profit/losses, the TensorFlow module cleverly decides how to adjust the weights in a way that I-dont-really-need-to-know, but by the time it has done all passes through the dataset V, it comes out with the hopefully 'optimal' weights. 6) If I am able train the weights on a test set, and validate them on data that the model hasnt seen before, then the model is effective and not just trained to the data it has seen.

Please would it be possible for someone to verify or correct the steps/understanding that I have described above? In particular, how is it possible to send feedback to the algorithm if the results of actions aren't known yet, and how would I know that Softmax is the right model to use?

Kind regards and thank you to anyone who can help me - any assistance would be much appreciated.


  • $\begingroup$ You need to read up on what's reinforcement learning versus the standard supervised learning. What you've outlined here is just the old fashion supervised learning. $\endgroup$ – horaceT Jan 20 '18 at 21:38
  • $\begingroup$ Thanks for the response @horaceT. I think this is where I'm having a bit of a meltdown as I can't work out how the learning algorithm can decide what to adjust in its policy if the outcome of the action it has just taken isn't known for several timesteps. Do you know of any good reading material/books/papers that might have tutorials/discussions/examples etc that might be simple enough for me to follow? Thank you again for your comment. $\endgroup$ – Paul Jan 22 '18 at 2:59
  • $\begingroup$ To formulate it as a RL problem, you need to ask youself, what is the MDP? The three components (reward, state, action), what are they? If you can't answer these satisfactorily, you're using the wrong approach. $\endgroup$ – horaceT Jan 22 '18 at 15:52
  • $\begingroup$ In my case, s (the state) is the price/technical analysis data available from the market, together with other metrics that I include. a (the action) is my choice at each timestep to buy, sell or do nothing. To me, the state (according to my definition above) is independant of the action taken. I.e. if my model pretends that I have infinite bankroll to simply things, then s' (the state after the action is taken) doesn't depend on a. Only the reward depends on a. $\endgroup$ – Paul Jan 23 '18 at 6:48
  • $\begingroup$ Apologies for double-comment. My situation satisfies the "Markov Property" in that what trades I have/haven't done (i.e. action taken) at any particular time doesn't influence the state or actions in future, so the process is "memoryless". (?) :) $\endgroup$ – Paul Jan 23 '18 at 6:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.