is this a time homogeneous markov chain

Question

Let $S=\{1,2\}$, $\omega_1, \dots, \omega_n, \dots$ are i.i.d discrete random variable taking values in $S$, consider $f_1, f_2 :\mathbb R_{+}\to \mathbb R_{+}$ as

$f_1(x)=\begin{cases} \frac{3x}{2} & x\leq 1 \\ \frac{1}{2}+\sqrt{x} & x>1 \\ \end{cases} $

$f_2(x)=\begin{cases} \frac{x}{4}+\frac{3}{4} & x\leq 1 \\ \sqrt{x} & x>1 \\ \end{cases} $

$p_k(1)=\frac{1}{2}+\frac{1}{2^k}, p_k(2)=\frac{1}{2}-\frac{1}{2^k}$

I start from some $X(0)=x(0)>0$ and choose $\omega_1$ according to probability distribution $p_1$ defined above and hence immediately get $f_{\omega_1}$.

Define $X(1)=f_{\omega_1}(X(0))$, repeat this 9 times and when chose $\omega_{10}$ according to probability distribution $p_2$ and define $X(10) =f_{\omega_{10}}(X(9))$ and repeat this iteration in this way and change the probability distribution again at $20^{th}, 30^{th}, \dots, 10k^{th}$ step $k=1,2,\dots$ according to probability dist $p_k$, continue this process.

And note that, from step 0 to step 9, we have $\mathbb P( \omega_i= j)= p_1(j), 1\le i\le 9, 1\le j\le 2$, similarly from step 10 to step 19, $\mathbb P( \omega_i= j)= p_2(j), 10\le i\le 19, 1\le j\le 2$, and so on.

(1) Could any-one tell me whether this is a Markov chain( time-inhomogeneous)? I understand that until the 49th iteration it is a Markov chain but I am confused as at the $50^{th}$ iteration probability distribution changes, does this means this forms an inhomogeneous Markov chain over-all?

(2) Any idea of how to conclude if there exists a stationary probability distribution $\pi$ such that $\mathbb P(X(n)\in \mathcal A)\rightarrow \pi(\mathcal A)$ as $n\to \infty$ for any Borel subset $\mathcal A$. Thanks.

(3) What if $f_1, f_2 :\mathbb R\to \mathbb R$ both are Lipschitz function with Lipschitz constant $<1$? Thanks

Answer 1

Markovian

Each $k-th$ step you refresh transitions by drawing a new $\omega_k$. So there is no history beyond the current value $x_{k-1}$ on which the new value $x_k$ depends. Therefore this process will be Markovian.

It is however not homogeneous, you do not have that the transition probabilities are the same in time $P(X_{k+1} \vert X_{k})$ is not independent of $k$.

---

Stable distribution could maybe be defined for a $\sigma$-algebra

Regarding the stable probability distribution, this is not easy to describe.

The functions $f_1$ and $f_2$ will capture the value of $x$ in between $1$ and $1 + \sqrt{3/4}$. The function $f_2$ will attract $x$ to $1$ and the function $f_1$ will attract $x$ to $1 + \sqrt{3/4}$.

You do get that $x$ will be, with increasing high probability, constrained to some small region, but I imagine that the set of points that $x$ occupies with some probability will not be stable (e.g. it may be questionable whether repeating $f_1$ and $f_2$ will ever lead to a cycle where the value $x$ repeatedly occurs).

Still, in terms of events $P(x-dx<X<x+dx)$ you might possibly describe some stable probability for them (if it exist).

The transformation $x \to 0.5 \sqrt{x}$ or $x \to \sqrt{x}$, each with 0.5 probability means that the density $f(x)$ in must relate to a sum of the densities $f((x-0.5)^2)$ and $f(x^2)$.

$$f(x) = \begin{cases} x f(x^2) &\quad \text{if}\quad 1 \leq x< 0.5 + \sqrt{3/4}\\ (x-0.5) f((x-0.5)^2) + x f(x^2) &\quad \text{if} \quad 0.5 + \sqrt{3/4} \leq x\leq 1.5\\ (x-0.5) f((x-0.5)^2) &\quad \text{if}\quad 1.5 < x \leq 1 +\sqrt{3/4} \end{cases}$$

I believe that there is a question here that asks whether there is a variable such that $X \sim -X/2$. That is a bit related. (In that case the answer is no).

---

Simulation

f1 <- function(x) {
  if (x<=1) {
    result = x*3/2
  } else {
    result = sqrt(x)+0.5
  }
  return(result)
}

f2 <- function(x) {
  if (x<=1) {
    result = (x+3)/4
  } else {
    result = sqrt(x)
  }
  return(result) 
}


run_x <- function(x_s = 0.25, n = 2*10^3) {
  x <- x_s
  p <- 1
  for (i in 1:n) {
    ### compute inhomogenoeus transition probabilities and sample omega
    k <- floor(i/10)+1
    p <- c(0.5+0.5^k,0.5-0.5^k)
    omega <- sample(c(1,2), 1, prob = p)
    
    ### evolve x according to omega
    if (omega == 1) {
      x <- c(x,f1(tail(x,1)))
    }
    if (omega == 2) {
      x <- c(x,f2(tail(x,1)))
    }
  }
  return(x)
}

plot(run_x(), pch = 21, 
     col = rgb(0,0,0,0.01), bg = rgb(0,0,0,0.01), cex = 0.5 ,
     ylim = c(1,1+sqrt(3/4)),
     xlab = "k", ylab = "x(k)",
     main = "results from 60 simulations")

for (i in 1:60) {
points(run_x(), pch = 21, 
       col = rgb(0,0,0,0.01), bg = rgb(0,0,0,0.01), cex = 0.5 )
}

x <- run_x(n=10^5)
h <- hist(x, breaks = seq(0,2, 0.001), xlim = c(1,1+sqrt(3/4)))