New answers tagged

3

Great that you have the formula written explicitly $$Y_{n + 1} = \begin{cases} \max\{ 0, A_n - K \}, & X_n = 0 \\ \max\{0, X_n - 1 + A_{n + 1} - K\}, & X_n > 0 \end{cases}.$$ Here $K=4$, since we have $4$ slots. Let's compute the conditional expected value of $Y_{n+1}$ given $X_n$. If $X_n=0$, then $E[Y_{n+1}|X_n=0]=E[\max\{0, A_n-4\}|X_n=0]=...


0

Put transition matrix into R. Let 1=Sranton, 2=Chi, 3=Phil, 4=NY, and 5=Seattle. Raise the transition matrix to a high power: P = (1/8)*matrix(c(4,1,1,1,1, 3,3,0,0,2, 3,0,1,4,0, 0,0,0,8,0, 0,0,0,0,8), byrow=T, nrow=5) P; P2 = P%*%P; P4 = P2%*%P2; P8 = P4%*%P4; P8 P16 = P8%*%P8; ...


1

The simplest proof here is just to combine states $3$ and $4$ to treat them as a single state. Combining these two states into a single state (called, say, $3\text{-}4$) does not remove the Markov property, so we have: $$\mathbb{P}(X_2 = 6 | X_1 = 3\text{-}4, X_0 = 2) = \mathbb{P}(X_2 = 6 | X_1 = 3\text{-}4).$$ This follows directly from the Markov ...


1

Starting from state $i$, draw a sample according to the distribution defined by that column and update the state according to the output. Here's an implementation in Python using $0$-index. import numpy as np A = np.array([[.25, .2, .25, .3], [.2, .3, .25, .3], [.25, .2, .4, .1], [.3, .3, .1, .3]]) xk = np.arange(len(A)) def generate_sample(cur_state): ...


2

It's called as stationary distribution and found via solving the equation: $$\pi=\pi \mathcal P$$ which is the left eigenvector of $\mathcal P$ corresponding to eigenvalue $\lambda=1$. Or, it's the usual (right) eigenvector of $\mathcal P^T$. After finding the eigenvector, negate the vector if elements are negative and normalise the vector such that it sums ...


3

Notation. Let $\pi = P(\text{Disease})$ be the prevalence of the disease in the population and $\tau = P(\text{Pos Test})$ be the proportion testing positive. For the test, let $\eta = P(\text{Pos}|\text{Disease})$ be the sensitivity and $\theta = P(\text{Neg}|\text{No Disease})$ be its specificity. Also, given test results, let $\gamma = P(\text{Disease}| ...


0

Your second question is easier. In the limit, $\frac{n\pm 1}{n} \rightarrow 1$, so $$\frac{\delta_{ij}}{n} + \dfrac{1}{2} \frac{(n + 1)p_{ij}}{n} + \dfrac{1}{2}\frac{(n - 1) p^{(2)}_{ij}}{n}$$ goes to $\frac{1}{2}p_{ij} + \frac{1}{2}p^{(2)}_{ij}$. For the first question: every time you go through an odd-numbered state, you increment by $p_{ij}$. This is ...


0

We have $$X_{n + 1} =\begin{cases} X_n - D_n & \text{if} X_n - D_n \ge 2 \\ 5 & \text{if} X_n - D_n \le 1 \end{cases} \tag{*}$$ The two rules of $(*)$ means no restocking and restocking respectively. We want $X_n$ to take value $i$ and $X_{n+1}$ to take value $j$ where $i,j \in \{2,3,4,5\}$. From $(*)$, it is natural to handle the case where $j$ ...


2

We can solve linear system of equations. Equation $(a)$ can be converted to $$(0.2-1)\pi_1 + 0.1\pi_2 + 0.55\pi_3=0\tag{a'}$$ Similarly for $b$ and $c$. Also, with the constraint $\pi_1+\pi_2+\pi_3=1$ We have $3$ variables and $4$ constraints. $$\pi=P^T\pi$$ $$e^T\pi=1$$ $$\begin{bmatrix} P^T-I \\ e^T\end{bmatrix}\pi =\begin{bmatrix} 0_3 \\ 1\end{...


1

If you understand Lemma 2 and Equation (6), the rest is just linearity of expectation: $$\begin{align}m_{ij}(n)&=E[N_n(j)|X_0=i]=E\left[\sum_{m=0}^n I_m(j)\bigg\vert X_0=i\right]=\sum_{m=0}^n E[I_m(j)|X_0=i]\\&=\sum_{m=0}^np_{ij}^{(m)}\end{align}$$ Intuitively, $p_{ij}^{(m)}$ represents how certain we're at $j$-th state at our $m$-th step. If it's $...


1

I think it's a notational issue. In your source, transition matrix sum up to $1$ columnwise. So, transitioning from a sunny day to rainy day, i.e. $P(X_k=3|X_{k-1}=1)$ is the entry at the third row and the first column of the seven times exponentiated transition matrix, i.e. $p_{13}^7=(P^7)_{31}$.


1

I believe it has to do with the Markovian assumption. It is assumed that $$P(X_n \mid X_{n-1}, \dots , X_0) = P(X_n \mid X_{n-1})$$ This is also true for larger lags $$P(X_n \mid X_{n-k}, X_{n-k-1}, \dots, X_0) = P(X_n \mid X_{n-k})$$


0

I found an analogous, less contrived, version of this example (in fact, based on the completeness and coherency of the example, it's probably the original version) from the textbook Introduction to Modeling and Analysis of Stochastic Systems, second edition, by Kulkarni: Example 2.8. (Telecommunications). The Tel-All Switch Corporation manufactures ...


0

If you are talking about the first equality: $P(D_n \ge i - 1) = 1 - P(D_n \le i - 2)$ $D_n$ is Poisson distributed, so it's discrete. Therefore the probability of the RV being greater than or equal to some value is simply just $1$ minus it's CDF of the discrete value just before it (i.e. $i-2$). Perhaps it is easier to see if you just move the $1$ to it'...


0

This property is a formula that says: "Arrivals to a full buffer are lost". However, there seems to be something missing here to make everything correct, it seems to me that this should read, $$ X_{n+1} = \text{min}\left(X_{n} + A_{n+1}, K\right) $$ That is, the number of items at time n+1 equals the previous number of items plus the number of arrivals, or ...


Top 50 recent answers are included