# is this a time homogeneous markov chain

Let $$S=\{1,2\}$$, $$\omega_1, \dots, \omega_n, \dots$$ are i.i.d discrete random variable taking values in $$S$$, consider $$f_1, f_2 :\mathbb R_{+}\to \mathbb R_{+}$$ as

$$f_1(x)=\begin{cases} \frac{3x}{2} & x\leq 1 \\ \frac{1}{2}+\sqrt{x} & x>1 \\ \end{cases}$$

$$f_2(x)=\begin{cases} \frac{x}{4}+\frac{3}{4} & x\leq 1 \\ \sqrt{x} & x>1 \\ \end{cases}$$

$$p_k(1)=\frac{1}{2}+\frac{1}{2^k}, p_k(2)=\frac{1}{2}-\frac{1}{2^k}$$

I start from some $$X(0)=x(0)>0$$ and choose $$\omega_1$$ according to probability distribution $$p_1$$ defined above and hence immediately get $$f_{\omega_1}$$.

Define $$X(1)=f_{\omega_1}(X(0))$$, repeat this 9 times and when chose $$\omega_{10}$$ according to probability distribution $$p_2$$ and define $$X(10) =f_{\omega_{10}}(X(9))$$ and repeat this iteration in this way and change the probability distribution again at $$20^{th}, 30^{th}, \dots, 10k^{th}$$ step $$k=1,2,\dots$$ according to probability dist $$p_k$$, continue this process.

And note that, from step 0 to step 9, we have $$\mathbb P( \omega_i= j)= p_1(j), 1\le i\le 9, 1\le j\le 2$$, similarly from step 10 to step 19, $$\mathbb P( \omega_i= j)= p_2(j), 10\le i\le 19, 1\le j\le 2$$, and so on.

(1) Could any-one tell me whether this is a Markov chain( time-inhomogeneous)? I understand that until the 49th iteration it is a Markov chain but I am confused as at the $$50^{th}$$ iteration probability distribution changes, does this means this forms an inhomogeneous Markov chain over-all?

(2) Any idea of how to conclude if there exists a stationary probability distribution $$\pi$$ such that $$\mathbb P(X(n)\in \mathcal A)\rightarrow \pi(\mathcal A)$$ as $$n\to \infty$$ for any Borel subset $$\mathcal A$$. Thanks.

(3) What if $$f_1, f_2 :\mathbb R\to \mathbb R$$ both are Lipschitz function with Lipschitz constant $$<1$$? Thanks

• Does this procedure then repeata (I. E. At the 100th observation you go back to using p1?) Nov 19, 2020 at 12:29

## 1 Answer

### Markovian

Each $$k-th$$ step you refresh transitions by drawing a new $$\omega_k$$. So there is no history beyond the current value $$x_{k-1}$$ on which the new value $$x_k$$ depends. Therefore this process will be Markovian.

It is however not homogeneous, you do not have that the transition probabilities are the same in time $$P(X_{k+1} \vert X_{k})$$ is not independent of $$k$$.

### Stable distribution could maybe be defined for a $$\sigma$$-algebra

Regarding the stable probability distribution, this is not easy to describe.

The functions $$f_1$$ and $$f_2$$ will capture the value of $$x$$ in between $$1$$ and $$1 + \sqrt{3/4}$$. The function $$f_2$$ will attract $$x$$ to $$1$$ and the function $$f_1$$ will attract $$x$$ to $$1 + \sqrt{3/4}$$.

You do get that $$x$$ will be, with increasing high probability, constrained to some small region, but I imagine that the set of points that $$x$$ occupies with some probability will not be stable (e.g. it may be questionable whether repeating $$f_1$$ and $$f_2$$ will ever lead to a cycle where the value $$x$$ repeatedly occurs).

Still, in terms of events $$P(x-dx you might possibly describe some stable probability for them (if it exist).

The transformation $$x \to 0.5 \sqrt{x}$$ or $$x \to \sqrt{x}$$, each with 0.5 probability means that the density $$f(x)$$ in must relate to a sum of the densities $$f((x-0.5)^2)$$ and $$f(x^2)$$.

$$f(x) = \begin{cases} x f(x^2) &\quad \text{if}\quad 1 \leq x< 0.5 + \sqrt{3/4}\\ (x-0.5) f((x-0.5)^2) + x f(x^2) &\quad \text{if} \quad 0.5 + \sqrt{3/4} \leq x\leq 1.5\\ (x-0.5) f((x-0.5)^2) &\quad \text{if}\quad 1.5 < x \leq 1 +\sqrt{3/4} \end{cases}$$

I believe that there is a question here that asks whether there is a variable such that $$X \sim -X/2$$. That is a bit related. (In that case the answer is no).

### Simulation

f1 <- function(x) {
if (x<=1) {
result = x*3/2
} else {
result = sqrt(x)+0.5
}
return(result)
}

f2 <- function(x) {
if (x<=1) {
result = (x+3)/4
} else {
result = sqrt(x)
}
return(result)
}

run_x <- function(x_s = 0.25, n = 2*10^3) {
x <- x_s
p <- 1
for (i in 1:n) {
### compute inhomogenoeus transition probabilities and sample omega
k <- floor(i/10)+1
p <- c(0.5+0.5^k,0.5-0.5^k)
omega <- sample(c(1,2), 1, prob = p)

### evolve x according to omega
if (omega == 1) {
x <- c(x,f1(tail(x,1)))
}
if (omega == 2) {
x <- c(x,f2(tail(x,1)))
}
}
return(x)
}

plot(run_x(), pch = 21,
col = rgb(0,0,0,0.01), bg = rgb(0,0,0,0.01), cex = 0.5 ,
ylim = c(1,1+sqrt(3/4)),
xlab = "k", ylab = "x(k)",
main = "results from 60 simulations")

for (i in 1:60) {
points(run_x(), pch = 21,
col = rgb(0,0,0,0.01), bg = rgb(0,0,0,0.01), cex = 0.5 )
}

x <- run_x(n=10^5)
h <- hist(x, breaks = seq(0,2, 0.001), xlim = c(1,1+sqrt(3/4)))

• @Marso the sub-chains may indeed be considered Markovian, but what I meant was that the entire process can be considered Markovian when you take steps of 50. If you consider transitions from {X(0+50k),.....,X(49+50k)} to {X(0+50(k+1)),.....,X(49+50(k+1)} or transitions from X(50k) to X(50(k+1)) then you have a Markovian process. Nov 20, 2020 at 6:46
• @Marso, it is still not a very clear question. It may possibly help as well to explain what sort of application, intuition or motivation is behind the question. Nov 20, 2020 at 12:39
• The way I understand it now is that you have a variable $X(k)$ that evolves according to $f_1$ with some probability $p_k$ and $f_2$ with some probability $1-p_k$ and these probabilities themselves are random variables that are changing each $j\cdot10$ -th step. Nov 20, 2020 at 12:46
• @Marso I have understood that the $\omega_1 = \omega_2 = \omega _ 3$ and so on for each 10 steps. Or are you changing $\omega_k$ for every step, and only the transition matrix according to how you change the $\omega_k$ changes every 10 steps? Nov 20, 2020 at 15:49
• @Marso with your initial explanation it was not so clear whether the $\omega_k$ change each step or every 10 steps. That makes the difference with my previous answer. Nov 20, 2020 at 16:58