Let $S=\{1,2\}$, $\omega_1, \dots, \omega_n, \dots$ are i.i.d discrete random variable taking values in $S$, consider $f_1, f_2 :\mathbb R_{+}\to \mathbb R_{+}$ as

$f_1(x)=\begin{cases} \frac{3x}{2} & x\leq 1 \\ \frac{1}{2}+\sqrt{x} & x>1 \\ \end{cases} $

$f_2(x)=\begin{cases} \frac{x}{4}+\frac{3}{4} & x\leq 1 \\ \sqrt{x} & x>1 \\ \end{cases} $

$p_k(1)=\frac{1}{2}+\frac{1}{2^k}, p_k(2)=\frac{1}{2}-\frac{1}{2^k}$

I start from some $X(0)=x(0)>0$ and choose $\omega_1$ according to probability distribution $p_1$ defined above and hence immediately get $f_{\omega_1}$.

Define $X(1)=f_{\omega_1}(X(0))$, repeat this 9 times and when chose $\omega_{10}$ according to probability distribution $p_2$ and define $X(10) =f_{\omega_{10}}(X(9))$ and repeat this iteration in this way and change the probability distribution again at $20^{th}, 30^{th}, \dots, 10k^{th}$ step $k=1,2,\dots$ according to probability dist $p_k$, continue this process.

And note that, from step 0 to step 9, we have $\mathbb P( \omega_i= j)= p_1(j), 1\le i\le 9, 1\le j\le 2$, similarly from step 10 to step 19, $\mathbb P( \omega_i= j)= p_2(j), 10\le i\le 19, 1\le j\le 2$, and so on.

(1) Could any-one tell me whether this is a Markov chain( time-inhomogeneous)? I understand that until the 49th iteration it is a Markov chain but I am confused as at the $50^{th}$ iteration probability distribution changes, does this means this forms an inhomogeneous Markov chain over-all?

(2) Any idea of how to conclude if there exists a stationary probability distribution $\pi$ such that $\mathbb P(X(n)\in \mathcal A)\rightarrow \pi(\mathcal A)$ as $n\to \infty$ for any Borel subset $\mathcal A$. Thanks.

(3) What if $f_1, f_2 :\mathbb R\to \mathbb R$ both are Lipschitz function with Lipschitz constant $<1$? Thanks

  • $\begingroup$ Does this procedure then repeata (I. E. At the 100th observation you go back to using p1?) $\endgroup$
    – Three Diag
    Nov 19, 2020 at 12:29

1 Answer 1



Each $k-th$ step you refresh transitions by drawing a new $\omega_k$. So there is no history beyond the current value $x_{k-1}$ on which the new value $x_k$ depends. Therefore this process will be Markovian.

It is however not homogeneous, you do not have that the transition probabilities are the same in time $P(X_{k+1} \vert X_{k})$ is not independent of $k$.

Stable distribution could maybe be defined for a $\sigma$-algebra

Regarding the stable probability distribution, this is not easy to describe.

The functions $f_1$ and $f_2$ will capture the value of $x$ in between $1$ and $1 + \sqrt{3/4}$. The function $f_2$ will attract $x$ to $1$ and the function $f_1$ will attract $x$ to $1 + \sqrt{3/4}$.

You do get that $x$ will be, with increasing high probability, constrained to some small region, but I imagine that the set of points that $x$ occupies with some probability will not be stable (e.g. it may be questionable whether repeating $f_1$ and $f_2$ will ever lead to a cycle where the value $x$ repeatedly occurs).

Still, in terms of events $P(x-dx<X<x+dx)$ you might possibly describe some stable probability for them (if it exist).

The transformation $x \to 0.5 \sqrt{x}$ or $x \to \sqrt{x}$, each with 0.5 probability means that the density $f(x)$ in must relate to a sum of the densities $f((x-0.5)^2)$ and $f(x^2)$.

$$f(x) = \begin{cases} x f(x^2) &\quad \text{if}\quad 1 \leq x< 0.5 + \sqrt{3/4}\\ (x-0.5) f((x-0.5)^2) + x f(x^2) &\quad \text{if} \quad 0.5 + \sqrt{3/4} \leq x\leq 1.5\\ (x-0.5) f((x-0.5)^2) &\quad \text{if}\quad 1.5 < x \leq 1 +\sqrt{3/4} \end{cases}$$

I believe that there is a question here that asks whether there is a variable such that $X \sim -X/2$. That is a bit related. (In that case the answer is no).




f1 <- function(x) {
  if (x<=1) {
    result = x*3/2
  } else {
    result = sqrt(x)+0.5

f2 <- function(x) {
  if (x<=1) {
    result = (x+3)/4
  } else {
    result = sqrt(x)

run_x <- function(x_s = 0.25, n = 2*10^3) {
  x <- x_s
  p <- 1
  for (i in 1:n) {
    ### compute inhomogenoeus transition probabilities and sample omega
    k <- floor(i/10)+1
    p <- c(0.5+0.5^k,0.5-0.5^k)
    omega <- sample(c(1,2), 1, prob = p)
    ### evolve x according to omega
    if (omega == 1) {
      x <- c(x,f1(tail(x,1)))
    if (omega == 2) {
      x <- c(x,f2(tail(x,1)))

plot(run_x(), pch = 21, 
     col = rgb(0,0,0,0.01), bg = rgb(0,0,0,0.01), cex = 0.5 ,
     ylim = c(1,1+sqrt(3/4)),
     xlab = "k", ylab = "x(k)",
     main = "results from 60 simulations")

for (i in 1:60) {
points(run_x(), pch = 21, 
       col = rgb(0,0,0,0.01), bg = rgb(0,0,0,0.01), cex = 0.5 )

x <- run_x(n=10^5)
h <- hist(x, breaks = seq(0,2, 0.001), xlim = c(1,1+sqrt(3/4)))
  • $\begingroup$ @Marso the sub-chains may indeed be considered Markovian, but what I meant was that the entire process can be considered Markovian when you take steps of 50. If you consider transitions from {X(0+50k),.....,X(49+50k)} to {X(0+50(k+1)),.....,X(49+50(k+1)} or transitions from X(50k) to X(50(k+1)) then you have a Markovian process. $\endgroup$ Nov 20, 2020 at 6:46
  • $\begingroup$ @Marso, it is still not a very clear question. It may possibly help as well to explain what sort of application, intuition or motivation is behind the question. $\endgroup$ Nov 20, 2020 at 12:39
  • $\begingroup$ The way I understand it now is that you have a variable $X(k)$ that evolves according to $f_1$ with some probability $p_k$ and $f_2$ with some probability $1-p_k$ and these probabilities themselves are random variables that are changing each $j\cdot10$ -th step. $\endgroup$ Nov 20, 2020 at 12:46
  • $\begingroup$ @Marso I have understood that the $\omega_1 = \omega_2 = \omega _ 3$ and so on for each 10 steps. Or are you changing $\omega_k$ for every step, and only the transition matrix according to how you change the $\omega_k$ changes every 10 steps? $\endgroup$ Nov 20, 2020 at 15:49
  • $\begingroup$ @Marso with your initial explanation it was not so clear whether the $\omega_k$ change each step or every 10 steps. That makes the difference with my previous answer. $\endgroup$ Nov 20, 2020 at 16:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.