0
$\begingroup$

I have the following question about the Absorption Times of Markov Chains in Continuous State-Space.

I was reading the following article on Absorption Times of Markov Chains (https://en.wikipedia.org/wiki/Absorbing_Markov_chain), but it only references Markov Chains in Discrete State-Space. I am interested in learning about Absorption Times of Markov Chains in Continuous State-Space. To illustrate my question, I thought of the following example (using the R programming language):

Suppose you have a Bivariate Normal Distribution with the following properties:

  • Mean = c(10, 10)

  • Sigma = matrix( c(1,0.5, 0.5, 1), # the data elements nrow=2, # number of rows ncol=2, # number of columns byrow = TRUE) # fill matrix by rows

I am interested in knowing "on average, how many (pairs of) random numbers do you need to generate from this Bivariate Normal Distribution until a value of of (12,12) is exceeded?

I tried to answer to this question using a computer simulation:

Sigma = matrix(
  c(1,0.5, 0.5, 1), # the data elements
  nrow=2,              # number of rows
  ncol=2,              # number of columns
  byrow = TRUE)        # fill matrix by rows

res <- matrix(0, nrow = 0, ncol = 3)    

for (j in 1:1000){
  e_i = data.frame(matrix(mvrnorm(n = 1, c(10,10), Sigma), ncol=2))
  i <- 1
  while(e_i$X1[1] < 12 | e_i$X2[1] < 12) {
    e_i = data.frame(matrix(mvrnorm(n = 1, c(10,10), Sigma), ncol=2))
    i <- i + 1
  }
  x <- c(e_i$X1, e_i$X2  ,i)
  res <- rbind(res, x)
}

res = data.frame(res)

The results of this simulation look as follows:

plot(hist(res$X3, breaks = 300))

enter image description here

And here is a summary of the simulation (On my own computer, I changed the number of iterations to 100,000 iterations):

summary(res$X3)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
    1.0    71.0   171.0   248.8   347.0  2259.0 

This tells us that on average, roughly 248 (pairs of) random numbers must be generated from this Bivariate Normal Distribution until a value of (12,12) is exceeded.

My Question: In the Wikipedia page I linked above, an example is shown on how to calculate the absorption times of a discrete Markov Chain for "the number of times you need to flip a coin until you get Heads, Tails, Heads":

enter image description here

In general, is there an analog of these calculations in Continuous State Space? For example, is there some formula that could have told me (prior to running the simulation) that a Bivariate Normal Distribution ~ [ mean(10,10) , var(1,0.5,0.5,1) ] would on average require 248 iterations before a value of (12,12) can be observed (i.e. the number of "steps" before absorption)?

Thanks!

$\endgroup$
3
$\begingroup$

Let's think about this for a moment. Continuous state space means that there are infinitely many states linked via transition probabilities such that they are all accessible by traversing a single graph (let's suppose). The expected values in such a case are asymptotic, meaning that they either have a cleverly constructed closed form (such as mean of a geometric distribution), or have to be approximated, usually via simulations, such as above. TLDR: There is no general expression for absorbing states in infinite state markov chains.

$\endgroup$
2
  • $\begingroup$ Thank you for your answer! In the end, would there have been a formula to obtain the answer 248? Thank you! $\endgroup$
    – stats555
    Nov 30 '21 at 1:55
  • $\begingroup$ Yes, there would be as the model (Bivariate Normal Distribution) is known apriori, and the sampling is i.i.d. $\endgroup$
    – hmi789
    Dec 1 '21 at 1:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.