How to use Markov Chain to calculate attribution worth of a state? I am trying to model a user's behavior through an app and have come across the idea of Markov Chains to do the modeling. A similar problem in marketing seems to be the multi-channel attribution problem. What I don't quite understand is how to attribute the positive outcome's result to each node along the way since my paths can be bidrectional.

In the image above, how do I attribute the purchase conversion to the Notification Center with markov chains? I've tried to see if there was a removal effect, but since the node itself doesn't touch the conversion path directly, it doesn't seem to change. Is the attribution of it just 0 then?
 A: To calculate the attribution of a node $N$ one can calculate the Removal Effect, i.e. the probability of a Purchase if the node $N$ "didn't exist" (= Exit note). This can be done by


*

*setting the outgoing transition probabilities of $N$ to 0 except for the transition to Exit, which is set to 1: $P(\text{Exit}|N) = 1$.

*calculate the resulting probability to purchase $P(\text{Purchase}|\text{Open App})$

*and comparing with the case where the node operates normally.


Let's do this...
Send Notification Center to Exit
The node transition matrix $\underline{P}$ now looks as follows:
$$
\scriptsize{
\begin{matrix}
{} &  Open App &  Social &  Store &  Notification Center &  Exit &  Purchase \\
Open App            &       0.0 &    0.00 &    0.0 &                  0.0 &   0.0 &       0.0 \\
Social              &       0.5 &    0.00 &    0.4 &                  0.0 &   0.0 &       0.0 \\
Store               &       0.5 &    0.30 &    0.0 &                  0.0 &   0.0 &       0.0 \\
Notification Center &       0.0 &    0.15 &    0.1 &                  0.0 &   0.0 &       0.0 \\
Exit                &       0.0 &    0.50 &    0.4 &                  1.0 &   1.0 &       0.0 \\
Purchase            &       0.0 &    0.05 &    0.1 &                  0.0 &   0.0 &       1.0 \\
\end{matrix}
}
$$
Calculate resulting probability of a purchase
Define $\vec \pi$ as the initial state vector (100%, 0, 0, 0, 0, 0) in the order of the above columns, i.e. at the start all the money is in the Open App node. Then we calculate the probabilities after a long time has passed (e.g. 1000 times steps). This yields $$\vec\pi \underline P^{1000} = (0, 0, 0, 0, 0.0, 88.6\%, 11.4\%)
$$
Compare with default case
This without sending everything from Notification Center directly to Exit, the chain yields $$\vec\pi \underline P_{default}^{1000} = (0, 0, 0, 0, 0.0, 86.3\%, 13.7\%)
$$

Accordingly, we can now argue that out of 100\$ that could potentially
  be spent in purchases, 2.3\$ would "get lost" if the Notification Center didn't exist (13.7\$ minus 11.4\$).

A: Since the attribution to the Store is the probability P(Purchase|Store), the corresponding attribution for the Notification Center is P(Purchase|Notification Center) and with the chain rule this can be decomposed as P(Purchase|Store)*P(Store|Notification Center) + P(Purchase|Social)*P(Social|Notification Center).
Plugging in the numbers thus yields 0.1*0.35+0.05*0.55 which is 0.0625 or 6.25%.
The nice things about Markov chains is exactly that you don't have to worry about loops, since the probabilities are always depending only on the previous state.
