Absorption Time of Continuous Markov Chains

Question

I have the following question about the Absorption Times of Markov Chains in Continuous State-Space.

I was reading the following article on Absorption Times of Markov Chains (https://en.wikipedia.org/wiki/Absorbing_Markov_chain), but it only references Markov Chains in Discrete State-Space. I am interested in learning about Absorption Times of Markov Chains in Continuous State-Space. To illustrate my question, I thought of the following example (using the R programming language):

Suppose you have a Bivariate Normal Distribution with the following properties:

• Mean = c(10, 10)

• Sigma = matrix( c(1,0.5, 0.5, 1), # the data elements nrow=2, # number of rows ncol=2, # number of columns byrow = TRUE) # fill matrix by rows

I am interested in knowing "on average, how many (pairs of) random numbers do you need to generate from this Bivariate Normal Distribution until a value of of (12,12) is exceeded?

I tried to answer to this question using a computer simulation:

Sigma = matrix(
  c(1,0.5, 0.5, 1), # the data elements
  nrow=2,              # number of rows
  ncol=2,              # number of columns
  byrow = TRUE)        # fill matrix by rows

res <- matrix(0, nrow = 0, ncol = 3)    

for (j in 1:1000){
  e_i = data.frame(matrix(mvrnorm(n = 1, c(10,10), Sigma), ncol=2))
  i <- 1
  while(e_i$X1[1] < 12 | e_i$X2[1] < 12) {
    e_i = data.frame(matrix(mvrnorm(n = 1, c(10,10), Sigma), ncol=2))
    i <- i + 1
  }
  x <- c(e_i$X1, e_i$X2  ,i)
  res <- rbind(res, x)
}

res = data.frame(res)

The results of this simulation look as follows:

plot(hist(res$X3, breaks = 300))

And here is a summary of the simulation (On my own computer, I changed the number of iterations to 100,000 iterations):

summary(res$X3)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
    1.0    71.0   171.0   248.8   347.0  2259.0 

This tells us that on average, roughly 248 (pairs of) random numbers must be generated from this Bivariate Normal Distribution until a value of (12,12) is exceeded.

My Question: In the Wikipedia page I linked above, an example is shown on how to calculate the absorption times of a discrete Markov Chain for "the number of times you need to flip a coin until you get Heads, Tails, Heads":

In general, is there an analog of these calculations in Continuous State Space? For example, is there some formula that could have told me (prior to running the simulation) that a Bivariate Normal Distribution ~ [ mean(10,10) , var(1,0.5,0.5,1) ] would on average require 248 iterations before a value of (12,12) can be observed (i.e. the number of "steps" before absorption)?

Thanks!

Answer 1

Let's think about this for a moment. Continuous state space means that there are infinitely many states linked via transition probabilities such that they are all accessible by traversing a single graph (let's suppose). The expected values in such a case are asymptotic, meaning that they either have a cleverly constructed closed form (such as mean of a geometric distribution), or have to be approximated, usually via simulations, such as above. TLDR: There is no general expression for absorbing states in infinite state markov chains.