# How to estimate loyalty to service?

Let's say there are two services $A$ and $B$ that solve same problem. We would like to know which one users more loyal to. Loyalty $L(X)$ is probability that user will use service $X$ when asked to solve problem.

Well for every user we can estimate $L$ for service $A$ like this $L(A) = \frac{N(A)}{N(A) + N(B)}$ where $N(X)$ is number of uses of $X$. We can even smooth it with Laplacian smoothing and get $L'(A) = \frac{N(A) + 1}{N(A) + N(B) + 2}$ but it's not the case.

In reality 20% of users have $N(A) + N(B) = 1$, then there are two types of $L(A)$: 0 and 1 both with variance like 0.5 =).

Maybe there are known methods how to behave in such situations when there was almost none experiments conducted. We need to have better estimation on both $L$ and it's variance.

-
"it's not the case" - I don't understand what you are trying to say here. Can you re-phrase? –  D.W. Jul 7 '12 at 23:26
I think smoothing does not solve my problem. That is what I tried to say –  alexanderkuk Jul 8 '12 at 8:54

As @MichaelChernick says, there is probably no good answer to reliably estimate the user's loyalty, for users who have not used any service very often.

(Arguably it would be reasonable to classify those users as not very loyal. You might decide that a user must use your service a minimum number of times before you're prepared to consider them a loyal user.)

One approach to dealing with these issues might be to use a Bayesian method. Compute the average loyalty, across all users. For a user who has no uses, the average loyalty is your best estimate. As the number of uses increases, your estimate should start to move in the direction of your $L(A)$ value.

In particular, the following might be a reasonable estimate: $$L_\text{est}(A) = \frac{n}{n+n^*} L(A) + \frac{n^*}{n+n^*} L^*(A),$$ where $n=N(A)+N(B)$ is the number of times this user has used a service, $n^*$ is the average value of $n$ across all users, $L(A)=N(A)/(N(A)+N(B))$ is your naive (uncorrected) loyalty ratio for this user, and $L^*(A)$ is the average value of $L(A)$ across all users.

If you know something about the characteristics of the user, and you think these characteristics will have a significant effect on user loyalty, you could perhaps take the average over all other users who are similar to this user, rather than over all users.

-
Thank you! I think we can play with definition of similar users. It can be: all users, user who made same amount of hits, user who made almost the same amount of hits, user of same gender and so forth. –  alexanderkuk Jul 8 '12 at 8:58
But will it improve estimate of variance for $L_{est}$? $L^*$ has to be preatty independant of $L$, doesn't it? –  alexanderkuk Jul 8 '12 at 9:02