How to use Markov Chain to calculate attribution worth of a state?

Question

I am trying to model a user's behavior through an app and have come across the idea of Markov Chains to do the modeling. A similar problem in marketing seems to be the multi-channel attribution problem. What I don't quite understand is how to attribute the positive outcome's result to each node along the way since my paths can be bidrectional.

In the image above, how do I attribute the purchase conversion to the Notification Center with markov chains? I've tried to see if there was a removal effect, but since the node itself doesn't touch the conversion path directly, it doesn't seem to change. Is the attribution of it just 0 then?

Answer 1

To calculate the attribution of a node $N$ one can calculate the Removal Effect, i.e. the probability of a Purchase if the node $N$ "didn't exist" (= Exit note). This can be done by

1. setting the outgoing transition probabilities of $N$ to 0 except for the transition to Exit, which is set to 1: $P(\text{Exit}|N) = 1$.
2. calculate the resulting probability to purchase $P(\text{Purchase}|\text{Open App})$
3. and comparing with the case where the node operates normally.

Let's do this...

Send Notification Center to Exit

The node transition matrix $\underline{P}$ now looks as follows:

$$ \scriptsize{ \begin{matrix} {} & Open App & Social & Store & Notification Center & Exit & Purchase \\ Open App & 0.0 & 0.00 & 0.0 & 0.0 & 0.0 & 0.0 \\ Social & 0.5 & 0.00 & 0.4 & 0.0 & 0.0 & 0.0 \\ Store & 0.5 & 0.30 & 0.0 & 0.0 & 0.0 & 0.0 \\ Notification Center & 0.0 & 0.15 & 0.1 & 0.0 & 0.0 & 0.0 \\ Exit & 0.0 & 0.50 & 0.4 & 1.0 & 1.0 & 0.0 \\ Purchase & 0.0 & 0.05 & 0.1 & 0.0 & 0.0 & 1.0 \\ \end{matrix} } $$

Calculate resulting probability of a purchase

Define $\vec \pi$ as the initial state vector (100%, 0, 0, 0, 0, 0) in the order of the above columns, i.e. at the start all the money is in the Open App node. Then we calculate the probabilities after a long time has passed (e.g. 1000 times steps). This yields $$\vec\pi \underline P^{1000} = (0, 0, 0, 0, 0.0, 88.6\%, 11.4\%) $$

Compare with default case This without sending everything from Notification Center directly to Exit, the chain yields $$\vec\pi \underline P_{default}^{1000} = (0, 0, 0, 0, 0.0, 86.3\%, 13.7\%) $$

Accordingly, we can now argue that out of 100\$ that could potentially be spent in purchases, 2.3\$ would "get lost" if the Notification Center didn't exist (13.7\$ minus 11.4\$).

Answer 2

Since the attribution to the Store is the probability P(Purchase|Store), the corresponding attribution for the Notification Center is P(Purchase|Notification Center) and with the chain rule this can be decomposed as P(Purchase|Store)*P(Store|Notification Center) + P(Purchase|Social)*P(Social|Notification Center).

Plugging in the numbers thus yields 0.1*0.35+0.05*0.55 which is 0.0625 or 6.25%.

The nice things about Markov chains is exactly that you don't have to worry about loops, since the probabilities are always depending only on the previous state.