Finding One Step Transition Matrix in Gambling?

Question

I need help finding what a one step transition matrix would look like for the following gambling scenario:

Using the bold strategy, say you have a certain amount of money x at any time and you're trying to get to y. Anytime 0 < x < 0.5y, you bet x amount of dollars. Any time 0.5y < x < y, you bet y-x dollars.

Assuming y = 20, how would I find the one step transition matrix?

I think I have a Markov state S on {0,1,2....20}, and 0 and 20 are absorbing states (you quit when you're broke or you quit when you hit the jackpot)...but I'm getting a little frustrated and having trouble on how to set it up as a transition matrix P.

I would greatly appreciate your help, and if possible, explanations.

Answer 1

for x=0(absorbing state)

for x=1 to 10, if the gambler losses, he moves down to 0(absorbed). But if he wins, he moves to 2x.

for x=11, if gambler losses, he moves down to 2, otherwise 20.

for x=12, if gambler losses, he moves down to 4, otherwise 20.

for x=13, if gambler losses, he moves down to 6, otherwise 20.

.........i.e. for x=11 to 20, if he losses, he moves down to 2(x-10). But if he wins, he moves to 20.

Let $p:$the gambler wins

transition matrix for 0,1,2,.........20.(shown for x=0,1,2,.....,19,20) $$\pmatrix{1 & 0&.&.&.&.&0 \\1-p & p\\1-p&.&.&p&.&.&0\\.\\\\.\\0 & 0&.&.&1-p(to~state~9)&.&p\\0 & 0&.&.&.&.&1}$$

Answer 2

We can generalise your problem by allowing any $y \in \mathbb{N}$ and assuming discrete states $x = 0,1,2...,y$. Assuming that gamble outcomes are independent with fixed win-probability $0 < \theta <1$ , the transition matrix $\mathbf{P}$ for the Markov chain has elements:

$$\mathbf{P}_{x_t,x_{t+1}}(y) = \begin{cases} \mathbb{I}(x_{t+1} = 0) & & \text{for } x_t = 0, \\[6pt] (1-\theta) \cdot \mathbb{I}(x_{t+1} = 0) + \theta \cdot \mathbb{I}(x_{t+1} = 2x_t) & & \text{for } x_t = 1,...,\lfloor y/2 \rfloor, \\[6pt] (1-\theta) \cdot \mathbb{I}(x_{t+1} = 2x_t-y) + \theta \cdot \mathbb{I}(x_{t+1} = y) & & \text{for } x_t = \lfloor y/2 \rfloor + 1,...,y-1. \\[6pt] \mathbb{I}(x_{t+1} = y) & & \text{for } x_t = y. \\[6pt] \end{cases}$$

The generalised transition matrix can be programmed in R as follows:

TRANS_MATRIX <- function(y, theta) {

    #Create transition matrix of zeros
    DIMNAMES <- 0:y;
    MATRIX   <- matrix(0, nrow = y+1, ncol = y+1, 
                       dimnames = list(DIMNAMES , DIMNAMES));

    #Set absorbing states
    MATRIX[1,1]     <- 1;
    MATRIX[y+1,y+1] <- 1;

    #Set states for lower gambles
    for (x in 1:floor(y/2)) { 
        MATRIX[x+1, 1]       <- 1-theta;
        MATRIX[x+1, 2*x+1]   <- theta; }

    #Set states for upper gambles
    for (x in (floor(y/2)+1):(y-1)) { 
        MATRIX[x+1, 2*x-y+1] <- 1-theta;
        MATRIX[x+1, y+1]     <- theta; }

    MATRIX }

Setting $y=20$ and $\theta = \tfrac{1}{2}$ gives the desired transition matrix, which we can display in R as follows:

#Generate transition matrix
MATRIX <- TRANS_MATRIX(20, 0.5);

#Print formatted matrix
print(format(MATRIX, nsmall = 1), quote = FALSE);

   0   1   2   3   4   5   6   7   8   9   10  11  12  13  14  15  16  17  18  19  20 
0  1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
1  0.5 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
2  0.5 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
3  0.5 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
4  0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
5  0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
6  0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
7  0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0
8  0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0
9  0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0
10 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5
11 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5
12 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5
13 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5
14 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5
15 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5
16 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5
17 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.5
18 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.5
19 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.5
20 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0