1
$\begingroup$

I need help finding what a one step transition matrix would look like for the following gambling scenario:

Using the bold strategy, say you have a certain amount of money x at any time and you're trying to get to y. Anytime 0 < x < 0.5y, you bet x amount of dollars. Any time 0.5y < x < y, you bet y-x dollars.

Assuming y = 20, how would I find the one step transition matrix?

I think I have a Markov state S on {0,1,2....20}, and 0 and 20 are absorbing states (you quit when you're broke or you quit when you hit the jackpot)...but I'm getting a little frustrated and having trouble on how to set it up as a transition matrix P.

I would greatly appreciate your help, and if possible, explanations.

$\endgroup$
1
$\begingroup$

for x=0(absorbing state)

for x=1 to 10, if the gambler losses, he moves down to 0(absorbed). But if he wins, he moves to 2x.

for x=11, if gambler losses, he moves down to 2, otherwise 20.

for x=12, if gambler losses, he moves down to 4, otherwise 20.

for x=13, if gambler losses, he moves down to 6, otherwise 20.

.........i.e. for x=11 to 20, if he losses, he moves down to 2(x-10). But if he wins, he moves to 20.

Let $p:$the gambler wins

transition matrix for 0,1,2,.........20.(shown for x=0,1,2,.....,19,20) $$\pmatrix{1 & 0&.&.&.&.&0 \\1-p & p\\1-p&.&.&p&.&.&0\\.\\\\.\\0 & 0&.&.&1-p(to~state~9)&.&p\\0 & 0&.&.&.&.&1}$$

$\endgroup$
1
$\begingroup$

We can generalise your problem by allowing any $y \in \mathbb{N}$ and assuming discrete states $x = 0,1,2...,y$. Assuming that gamble outcomes are independent with fixed win-probability $0 < \theta <1$ , the transition matrix $\mathbf{P}$ for the Markov chain has elements:

$$\mathbf{P}_{x_t,x_{t+1}}(y) = \begin{cases} \mathbb{I}(x_{t+1} = 0) & & \text{for } x_t = 0, \\[6pt] (1-\theta) \cdot \mathbb{I}(x_{t+1} = 0) + \theta \cdot \mathbb{I}(x_{t+1} = 2x_t) & & \text{for } x_t = 1,...,\lfloor y/2 \rfloor, \\[6pt] (1-\theta) \cdot \mathbb{I}(x_{t+1} = 2x_t-y) + \theta \cdot \mathbb{I}(x_{t+1} = y) & & \text{for } x_t = \lfloor y/2 \rfloor + 1,...,y-1. \\[6pt] \mathbb{I}(x_{t+1} = y) & & \text{for } x_t = y. \\[6pt] \end{cases}$$

The generalised transition matrix can be programmed in R as follows:

TRANS_MATRIX <- function(y, theta) {

    #Create transition matrix of zeros
    DIMNAMES <- 0:y;
    MATRIX   <- matrix(0, nrow = y+1, ncol = y+1, 
                       dimnames = list(DIMNAMES , DIMNAMES));

    #Set absorbing states
    MATRIX[1,1]     <- 1;
    MATRIX[y+1,y+1] <- 1;

    #Set states for lower gambles
    for (x in 1:floor(y/2)) { 
        MATRIX[x+1, 1]       <- 1-theta;
        MATRIX[x+1, 2*x+1]   <- theta; }

    #Set states for upper gambles
    for (x in (floor(y/2)+1):(y-1)) { 
        MATRIX[x+1, 2*x-y+1] <- 1-theta;
        MATRIX[x+1, y+1]     <- theta; }

    MATRIX }

Setting $y=20$ and $\theta = \tfrac{1}{2}$ gives the desired transition matrix, which we can display in R as follows:

#Generate transition matrix
MATRIX <- TRANS_MATRIX(20, 0.5);

#Print formatted matrix
print(format(MATRIX, nsmall = 1), quote = FALSE);

   0   1   2   3   4   5   6   7   8   9   10  11  12  13  14  15  16  17  18  19  20 
0  1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
1  0.5 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
2  0.5 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
3  0.5 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
4  0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
5  0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
6  0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
7  0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0
8  0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0
9  0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0
10 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5
11 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5
12 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5
13 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5
14 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5
15 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5
16 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5
17 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.5
18 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.5
19 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.5
20 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.