2
$\begingroup$

I am analyzing the WoW change in conversion rate (visitors who booked / visitors).

Let's say the WoW change from week 1 to week 2 was -10% (dropped 10%). Now, I want to know what/where this drop in conversion rate came from. I have a hunch/business knowledge that this drop is coming from northeastern states in the US. So, I remove all northeastern states from the data and check the WoW drop. I see, now the drop is just 1%. That means 90% of the WoW drop is coming from NE states. Does this make sense? Is it mathematically correct?

Second part of my question is, I don't get the same answer if I do the same process as above one state at a time. I don't understand why that is the case.

For example, let's say I remove NY first and see the change, it's 5% (so that's 50% of the drop coming from NY). Second, I remove New Jersey, and I get x%, third I remove Massachusetts and I get x%, and so on… If I repeat this process for all NE states and I add up the effect from each one of them, I get a total of 75% (lower than the 90% I got from the first method, where I removed all NE states at once).

Why this is the case? Any help to make me understand this is much appreciated. Note that visitors in one state is not in any other state. Thanks!

Improve this question
$\endgroup$
13
  • $\begingroup$ Hi: I think ( didn't check this but a simple example would probably show it ) you're one at a time thing isn't working because each state might have a different number of visitors booked, so, when you calculate the resulting "overall percent" drop, the denominator is different each time. you'd have to weight each state's drop by the number of people in the state if you wanted the "one state at a time" approach to give the same results as the "all at once" approach. $\endgroup$
    – mlofton
    Feb 25, 2021 at 0:52
  • $\begingroup$ Thanks . That makes sense. so, removing all states at once is the right approach ? If yes, how can I get the effect of individual states. Can I remove one state, then two, three , etc.. and see the incremental impact for each state ? $\endgroup$
    – bp0308
    Feb 25, 2021 at 2:04
  • 1
    $\begingroup$ When you measure the effect of each state additively rather than multiplicatively, everything will add up correctly. $\endgroup$
    – whuber ♦
    Feb 25, 2021 at 13:46
  • 1
    $\begingroup$ @bpo308: I think what huber is saying is to not use percentages. That's one good suggestion. Removing the states all at once ( in the percentage framework ) is mis-leading because of the possible differences in numbers of bookings. Think of the extreme case where NY has 1 million bookings and New jersey has 10. Then the effect might be due to NY and not the Northeast. So, if you accept the lack of additivity, then looking at individual percentage state effects is better info than adding the states up and then looking at percentages. So, I would stick with the individual state approach. $\endgroup$
    – mlofton
    Feb 25, 2021 at 14:27
  • $\begingroup$ Thanks !. @whuber How do I do this additively Vs. Multiplicatively ? Does additive mean; I remove NY first and let's say for example 50% of the WoW drop is from NY. Then I remove NY & NJ and see that 60% of the drop is from NY & NJ together. That means NY contrbutes 50% and NJ contributes 10% of the drop . Next I remove NY, NJ, MA And so on ... I can continue like this until reach close to a 100% of the WoW drop . Is this reasonable ? $\endgroup$
    – bp0308
    Feb 25, 2021 at 18:33

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.