Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.

I have a finite-state, time-independent Markov chain with two absorbing states which models educational outcomes (the absorbing states are completing and not completing). The transition probabilities are estimated by taking the proportions of people who move from one state to another at successive time points (based on a census of the population at two successive time points), and I have calculated the stationary vector.

However, since this needs to be done with several different cuts of the data, I would like to know if there is any way of associating a confidence interval to the entries of the stationary vector, to aid in identifying significant differences.

The article

Karson, M. J. and Wrobleski, W. J. (1976),
CONFIDENCE INTERVALS FOR ABSORBING MARKOV CHAIN PROBABILITIES APPLIED TO LOAN PORTFOLIOS.
Decision Sciences, 7: 10–17.
doi: 10.1111/j.1540-5915.1976.tb00653.x

looks helpful, but I'm not sure if it what I need. So my questions are:

  1. Is there a way to estimate confidence intervals for the stationary vector?

  2. If yes, and it is in the cited article, do I just need to grit my teeth and push through it, or is there a more modern treatment?

  3. (a long shot, but could save me some work) Is there is a macro or similar for SAS to estimate said confidence intervals?

share|improve this question
Since the Markov chain you consider has some absorbing states (and is irreducible, presumably), its stationary distribution is concentrated on these absorbing states. In other words, the stationary vector you computed should have exactly two non zero coordinates, one for each of the absorbing states. Is this really what you have in mind? – Did Jan 22 '11 at 0:13
Yes. The issue is computing some confidence intervals for said entries. – David Roberts Jan 24 '11 at 3:08
I'm asking the moderators to close this as no longer relevant. – David Roberts Jan 30 '11 at 22:39
@David Are you sure you want to remove it? I think it should stay because it still may be useful for other people. – mbq Jan 31 '11 at 10:38
I don't want to delete it, but close it (on MathOverflow there's a difference, but they don't run SE2.0). I'm happy to leave it as is for the sake of posterity, but I've realised my problem is not what I was asking in this question. – David Roberts Jan 31 '11 at 22:42
show 2 more comments

1 Answer

So, as said in the comments, the Markov chain you consider has some absorbing states (and is irreducible, presumably), hence its stationary distribution is concentrated on these absorbing states. Therefore the issue is to compute some confidence intervals for the only two non zero coordinates of the stationary vector, one for each of the absorbing states. I would call these entries absorption probabilities rather than stationary vector because there is not much really stationary here, but anyway...

Since there are two absorbing states, $a$ and $b$ say, you are interested in $u_c=P_c[$The chain is absorbed at $a]$, for a given initial state $c\ne a$, $b$. I gather you observe the number $N_t(x)$ of particles at state $x$ and time $t$, for every state $x$ (or every $x\ne a$, $b$?), at two different times $t_1$ and $t_2$. How to estimate $u_c$ from these counts? Surely I am missing the obvious but, even replacing (a multiple of) each $N_t(x)$ by the exact value of $P_c[$The chain is at $x$ at time $t]$, I do not see how to compute $u_c$ from these quantities.

Dimensional analysis shows that for $n$ states, $N_t(x)$ at every state $x$ and two different times yields $2(nāˆ’1)$ independent parameters and the transition matrix has $n(nāˆ’1)$ independent parameters, a fact which seems to indicate that it would be impossible to identify the latter from the former as soon as $n\ge3$. OK, this argument is too sloppy to be really conclusive but...

(Caveat: I did not read, and have no access to, the paper by Karson and Wrobleski.)

share|improve this answer
it is possible to estimate the probability of absorption in a given state given any initial distribution if you have the transition matrix. The entries of the transition matrix are estimated as you say, but you forget that in the presence of $m$ absorbing states you don't have $n(n-1)$ parameters but $(n-1)(n-m)$ (at least by my rough count), so for 4 states and 2 absorbing states, you have exactly enough parameters to estimate the transition matrix (and in my case there are further constraints which cut down the number of parameters needed - this is immaterial) – David Roberts Jan 30 '11 at 22:56
See e.g. math.furman.edu/~tlewis/math110/maki/chap8/sec4handout.pdf for a quick discussion of calculating absorption probabilities. – David Roberts Jan 30 '11 at 23:04
David: I was not aware that the situations you were interested in involved such small values of $n$ (and you are right, the number of parameters is $(n-1)(n-m)$). May I ask why you now want this question closed? Did you solve the question? I note that there is nothing in the handout you refer to about the estimation problem you asked. – Did Jan 31 '11 at 8:18
I've realised that what the paper cited shows is not really what I need. The paper deals with absorption probabilities calculated from samples, whereas I have complete information, and the uncertainty is not in how well I can guess the true values of the model given incomplete information, but the fit of the model to reality. – David Roberts Jan 31 '11 at 23:24

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.