Drunken cockroach - Trying to meet expected value

Question

Imagine that you have $1000 that you can split however you want. You bet in a cockroach run, but it is not the finish that's interesting. You can bet for the cockroach to go left or right, and you gain proportionally to how much he moved in or against your favour from the time you enter to the moment you decide to quit.

But you're smart. You have found that there are patterns, and our little hero tends to reach some areas with some probability before finally turning around and going in the other direction.

By the final turn, we mean that in his infinite race, he will not drift further to the left or right for the next 24h. He will either stick to the side or drift to the other side where the second is much more probable.

The blue and red histograms (normalized to 1.0) show the probability of the furthest positions that the cockroach reaches before he turns around (makes the final turn before he begins his long journey to reach the other side).

The black lines show the path of the cockroach.

From the histograms that show the statistically collected histogram of the extreme turning point, you can easily calculate the expected values for most extreme drift to the left or to the right.

You get paid 1 dollar for each millimeter that the cockroach rifts in your favour, and you lose 1 dollar when he goes 1 mm against you.

You decide when to enter and when to exit to collect gain or loss.

There are a few interesting optimization questions:

1. How should you enter your partial bets to enter for as close to $1000 as possible and to get as close to expected value as possible (I think that exceeding it is not possible)?

2. How should you proceed with quitting all you have managed to enter (probably you have not managed to enter with whole $1000)?

3. How to deal with misses? As far as missing enters is not a big deal, missing an exit point might lead to a complete disaster because the 'little puppy' is not going against you.

4. Should we need to measure some other properties of the path to make better bets as for the 'averaging' of entry / exit positions?

These questions puzzle me a lot, and finding a clue toward solving at least one will let me sleep better. :)

Answer 1

I'll try for an answer which hopefully goes into the right direction, but if not, just comment. Since you mentioned markets, I guess your cockroach corresponds simply to the price, whereas the allowed region is something like support and resistance. (Further, the your questions also remind me a bit about pricing of path-dependent options.) Your problem is probably described by some continuous stochastic model (such as implied by the Kolmogorov equation), but as this is probably not needed in practice I will present a simpler discrete model (such kind of models are called binomial/trinomial, Cox-Ross-Rubinstein or simply lattice methods in option pricing theory).

So, assume you have a grid $x_i$ which spans the allowed region (or does so at least with a good probability) and which has a grid-spacing of 1 mm. Let the resulting grid have $N$ gridpoints. Further, we are working at discrete and equally spaced time-points $t$. Suppose further, that the cockroach starts at time $t=0$ at gridpoint $i$.

Now consider a random walk on this grid. At each gridpoint $i$, you have a probability $r_i$ of going right and $l_i=1-r_i$ of going left. These probabilities need to be adjusted by you according to your model. Basically, on the left end of the grid the probability $r_i$ for stepping right must be significant, and vice versa.

If you have adjusted your model, you can easily evaluate numerically your expected gain and loss by simply simulating the random process (I didn't get exactly the scoring mechanism, but you can for instance assume that each correct direction gives you a factor $q$ on your invested capital, further there should be fixed costs for entering, etc.). I planned to add some formulas here, but as it is simple I'll let it be. Just simulate your random process multiple times and each time you are correct (incorrect) increase (decrease) your money. In the end take the average of all results (that is, do Monte-Carlo. You could alternatively use Dynamic Programming).

Further, in a next step, you can optimize the decisions (which decision in left/right and how much capital -- for the latter see the Kelly criterium). However, for this simple model, the conclusion will probably be not all too surprising. I guess you shall always enter with a bet in the opposite direction if the cockroach is on one end.

Summarizing, if you want to use the model for something useful, you should tune it to your needs. Still I hope this gives you an idea of how to approach a solution to your question.

---

Tuning the model

The above model assumes static and known transition probabilities and one-step up/down moves only. One obvious first extension, stated also in the comment to the original question, would be to remove the assumption that the probabilities are known. Your model then needs to perform some kind of stochastic optimization, i.e. at the same time estimate the probabilities (exploration) and make the best possible profit (exploitation). This model then has some simililarities with the famous multi-armed bandit problem. For the solution, you could then use the methods from Reinforcement Learning or Approximate Dynamic Programming. Although still simple, such a model could already be of practical relevance.

Other extensions:

• Instead of one-step up/down moves you could allow for arbitrary moves on the grid, each one with a probability to be estimated. As the problem might become quickly complex, you can for instance group the parameters (e.g. use common probabilities for gridpoints in the upper/middle/lower regions).
• Assume non-static but known (--well guessed) parameters (as suggested in the comment by @CarrKnight): here, the parameters are given, but you must estimate in which szenario or hidden state you are at the moment.
• Non-static, unknown parameters ... this one becomes hard, I guess. Simple models are required.

More ideas are welcome.