I am setting up a binary classification problem where if a trade is profitable it gets a 1, and if it is unprofitable, it gets a 0.

I have some independent variables, one of which is price_of_security. I have the dependent variable of trade_profit. I have data on about 20,000 such trades. I train a neural network on this data and then save the model. Next, I take 1 data point and make ten copies of it with increments of the price going down, and increments of the price going up, I hold all of the other independent variables constant. Like so:

|ticker|  price|
|   WTO| 216.94|
|   WTO| 217.94|
|   WTO| 218.94|
|   WTO| 219.94|
|   WTO| 220.94| 
|   WTO| 221.94| # This is the original price
|   WTO| 222.94|
|   WTO| 223.94|
|   WTO| 224.94|
|   WTO| 225.94|

I then put this data into my saved neural network and see if the neural network associated a lower price with a higher chance of it being a successful trade. I believe that it intuitively should associate a cheaper price with a higher chance of success and also associate higher prices with lower chances of success. My reasoning is that if the fair value of a security is 215, then my model should not recommend a buy at 220 and it should recommend a buy at 210. However, this is what the model predicts:

|ticker|  price| probability_1| label|
|   WTO| 216.94|          0.32|     0|
|   WTO| 217.94|          0.37|     0|
|   WTO| 218.94|          0.42|     0|
|   WTO| 219.94|          0.44|     0|
|   WTO| 220.94|          0.49|     0|
|   WTO| 221.94|          0.51|     1|
|   WTO| 222.94|          0.56|     1|
|   WTO| 223.94|          0.57|     1|
|   WTO| 224.94|          0.58|     1|
|   WTO| 225.94|          0.60|     1|

The model, however, finds that higher prices are associated with a higher chance of the trade being a 1 (aka a successful trade). I am trying to understand what is going on so I did a table of all the data I have to see what the model is learning. This is the table:

|    bin    |   num trades | ratio succes|
|   200-205 |           568|         0.85|
|   205-210 |           754|         0.79|
|   210-215 |          1200|         0.78|
|   215-220 |          4452|         0.55|
|   220-225 |          6783|         0.59|
|   225-230 |          8197|         0.68|

After a lot of thinking, I think the reason that I'm seeing the model associate a higher chance of success with a higher price is because there are so more (in total) successful trades in the higher-prices. Perhaps those trades make less money per trade, but the trained model only cares about the 0 or the 1, not about the magnitude of the profit. I think that is why I'm seeing this counter-intuitive results.

Should this experiment be set up as a regression problem rather than a classification problem?


You are probably picking up a spurious correlation because your input variable is not stationary. One common way of getting a (more) stationary series is to take the first differences.

Some leads: https://www.researchgate.net/post/Is_it_necessary_to_ensure_stationarity_of_all_time_series_variables_when_you_run_a_Vector_Autoregressive_VAR_Model

In this particular instance, one would expect such results if the price was stagnant for a long period of time below 221.94 and then rallied with minimal pullbacks after breaking that level.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.