# How Do I Know If A Markov Chain Follows The Markov Property?

In Probability and Statistics, a Markov Chain is said to have the "Markov Property" if the next state of this Markov Chain only depends on the current state of this Markov Chain.

This being said, I had the following question: Suppose I have some data about a Discrete Markov Process, such as the weather each day (e.g. Sun, Rain, Snow) over a period of days. I believe that my data has 1st Order Markov Property, i.e. the next day's weather only depends on the current weather. I go ahead and fit a 3-State Discrete Markov Chain to this data and estimate the transition probabilities corresponding to the 3 x 3 transition matrix. But how exactly do I determine if the data I have collected actually obeys the Markov Property?

The only thing that comes to mind are simulation methods - that is, if I create a Markov Model and then test this model, and the state sequence transitions reasonably match the weather in the real world, I could informally conclude that my data obeys the Markov Property. However, this seems informal and I was looking for a more "mathematically rigorous" way to determine if my data follows the Markov Property.

Could someone please comment on this?

Note: Apparently, there also exist the "2nd Order Markov Property" in which the next state of this Markov Chain only depends on the current and the previous state of this Markov Chain. I would be curious to know how we could test to see if the same data we collected earlier might obey the "2nd Order Markov Property".

• One way I can think of is to estimate $p(x_1,x_2,x_3)$ and $p(x_3 | x_2),p(x_2|x_1),p(x_1)$, and then make sure that $p(x_1,x_2,x_3) = p(x_3 | x_2)p(x_2|x_1)p(x_1)$ for every possible value of $(x_1,x_2,x_3)$. Jan 25, 2023 at 1:43

An initial and simple test for this would be to see if the data show evidence of the weather being affected by the weather two days ago when you're already conditioning on the weather one day ago. To do this you would form a $$3 \times 3 \times 3$$ contingency table for three consecutive days and use this to conduct a test of conditional independence to see if the Markov property is violated. If you find evidence that the weather two days ago is not independent of the weather today, conditional on the weather one day ago, that is a violation of the Markov property (and suggests that there is auto-correlation in the data that goes back more than one period).

You can easily test this by doing multinomial regression.

To fit the null hypothesis of a first order Markov chain you would include the previous state as a covariate in the model. You then estimate 6 parameters which translates into an estimate of the $$3\times3$$ transition matrix.

To fit the alternative hypothesis that you have a 2. order Markov chain, you instead include both the state at time $$t-1$$ and $$t-2$$ and an interaction between those as covariates. You then estimate 18 parameters which translates into an estimate of $$27$$ transition probabilities (from each of the $$3\times 3=9$$ second order states the chain can transition only to three possible states).

You can then test the null against the alternative using a likelihood ratio test in the usual way.

The following shows an R implementation of the above as well as the test of conditional independence proposed by @Ben via log-linear models. The resulting identical LRT-statistics suggest that the tests are equivalent.

# Simulate some data
n <- 100
set.seed(4)
x <- factor(sample(1:3, n, replace = TRUE))
y <- x[-(1:2)]
xlag1 <- x[-c(1,n)]
xlag2 <- x[-((n - 1):n)]

# Fit the null hypothesis
library(VGAM)
mod0 <- vglm(y ~ xlag1, family = multinomial())
predictvglm(mod0, newdata = data.frame(xlag1 = factor(1:3)), type = "response")
#>           1         2         3
#> 1 0.4166667 0.3055556 0.2777778
#> 2 0.3636364 0.3636364 0.2727273
#> 3 0.3103448 0.3448276 0.3448276

# Fit the alternative
mod1 <- update(mod0, .~xlag1*xlag2)
predictvglm(mod1, newdata = expand.grid(xlag1 = factor(1:3),xlag2 = factor(1:3)), type = "response")
#>           1          2         3
#> 1 0.2000000 0.46666667 0.3333333
#> 2 0.3636364 0.18181818 0.4545455
#> 3 0.3333333 0.33333333 0.3333333
#> 4 0.7500000 0.08333333 0.1666667
#> 5 0.3333333 0.50000000 0.1666667
#> 6 0.4444444 0.33333333 0.2222222
#> 7 0.3333333 0.33333333 0.3333333
#> 8 0.4000000 0.40000000 0.2000000
#> 9 0.1818182 0.36363636 0.4545455

# Test the null against the alternative
anova(mod0, mod1, test="LRT", type = "I")
#> Analysis of Deviance Table
#>
#> Model 1: y ~ xlag1
#> Model 2: y ~ xlag1 + xlag2 + xlag1:xlag2
#>   Resid. Df Resid. Dev Df Deviance Pr(>Chi)
#> 1       190     213.56
#> 2       178     198.24 12   15.322   0.2243

# Testing conditional independence as suggested Ben via log-linear models
tables <- table(y,xlag2,xlag1)
tables
#> , , xlag1 = 1
#>
#>    xlag2
#> y   1 2 3
#>   1 3 9 3
#>   2 7 1 3
#>   3 5 2 3
#>
#> , , xlag1 = 2
#>
#>    xlag2
#> y   1 2 3
#>   1 4 4 4
#>   2 2 6 4
#>   3 5 2 2
#>
#> , , xlag1 = 3
#>
#>    xlag2
#> y   1 2 3
#>   1 3 4 2
#>   2 3 3 4
#>   3 3 2 5
library(MASS)
loglin(tables, list(c(1,3),c(2,3)))
#> 2 iterations: deviation 0
#> $$lrt #> [1] 15.32195 #> #>$$pearson
#> [1] 14.73557
#>
#> $$df #> [1] 12 #> #>$$margin
#> $$margin[[1]] #> [1] "y" "xlag1" #> #>$$margin[[2]]
#> [1] "xlag2" "xlag1"
$$$$
`
• can you explain what you did here? i have a few questions about this Feb 3 at 5:51
• 1) why did you choose vgam here? could any regression model have been chosen? Feb 3 at 5:52
• 2) can you write the mathematical equations corresponding to the null hypothesis (markov property is satisfied) and the alternate hypothesis (markov property is not satisfied)? If I understand correctly, the interaction term is used as a proxy variable to show correlation? Feb 3 at 5:55
• 3) can you explain some of your R code? why did you use the update function instead of refitting the model via the vglm() function? and why is the expand.grid() function needed during the prediction? Feb 3 at 5:57
• 4) why are both ANOVA and the log-linear approach needed here? What conclusions can be made in this specific example based on the outputs of the ANOVA? Feb 3 at 5:59