1
$\begingroup$

I have data of how much a customer has spent with an app. The data looks something like this:

[1]  11.51  12.28  22.86  57.91  12.20   6.08  34.19  53.08 253.63  84.03  23.46   6.04   0.00
[14]   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00
[27]   0.00   0.00   0.00   0.00

There are 6267 observations in the data and only 157 values are non-zero. The histogram of this distribution is very heavily right-skewed with zero as the most frequent value.

enter image description here

Here is the summary of the data:

 Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
  0.0000   0.0000   0.0000   0.8794   0.0000 502.6000

and the standard deviation is:

sd(data$revenue)
[1] 10.56173

What I would like to do is model this distribution. I thought that I could model the distribution with a gamma curve but it appears that I cannot because of the zeros in the data.

fit <- fitdistr(data$revenue, "gamma")
Error in stats::optim(x = c(11.51, 12.28, 22.86, 57.91, 12.2, 6.08, 34.19,  : 
  initial value in 'vmmin' is not finite

I would also like to determine the sample size needed to calculate the mean of this data and build a confidence interval around that mean given this heavy right-skewed distribution.

My questions are:

  1. What is the best distribution to model this curve?

  2. How can I determine the sample size that I would need to calculate the mean with a 5% margin of error and a 95% confidence interval?

Any help or comments would be greatly appreciated! Thank you!

NOTE: For (2) I thought about building the sampling distribution for the mean of this distribution and getting mean and confidence interval of that distribution - however I am not sure how to estimate the sample size from that distribution.

Also, here is the data with zeros removed: enter image description here

|cite|improve this question
$\endgroup$
  • 1
    $\begingroup$ What is your data about? I see that you titled the plot "revenue", but is it revenue of what exactly? What are the zeros? I ask because finding appropriate distribution is not only about fitting the shape of it. For example: you could use some zero-inflated distribution. $\endgroup$ – Tim♦ Aug 30 '16 at 5:12
  • $\begingroup$ Hi Tim - The data I showed is just the revenue column but each element is the total monetary value of a unique customer. So for Example: The first element User1 has spend a total of \$11.51 and the last value User30 has spend a total of \$0.0 dollars. $\endgroup$ – RDizzl3 Aug 30 '16 at 5:17
  • 1
    $\begingroup$ There's no universally "best" distribution choice for a model; if you offer specific criteria there might be but it's unlikely to be anything simple. There are some plausible models. For example, if a gamma was a reasonable fit to the data that's >0, you could look at a zero-inflated gamma. $\endgroup$ – Glen_b♦ Aug 30 '16 at 5:27
  • $\begingroup$ @Glen_b - thank you for your comment. I think what I am interested in more than anything is determining a reliable confidence interval around the mean of this type of distribution - I was under the impression I should know the type of distribution in order to determine this. I would also like to be able to determine the sample size needed to also be confident the mean is reliable. I hope that makes sense. $\endgroup$ – RDizzl3 Aug 30 '16 at 6:16
  • $\begingroup$ Thanks. That's helpful in visualizing the non-zero information. However, your description still leaves many questions unanswered. Why is sales so fat-tailed, in other words, why is there such a wide range to it? Are sales unique to one and only one customer? Or are customers purchasing on multiple occasions? What is the relationship between sales and time? Do you have sales for multiple apps? What additional information is available to you, e.g., marketing activity, customer background factors, etc.? $\endgroup$ – DJohnson Sep 1 '16 at 20:32
2
$\begingroup$

  1. From what little data you show it looks like a gamma approximation for the positive values might be roughly reasonable with a Bernoulli for the zero/nonzero, though there may well be other choices. You could then perhaps use simulation to get an approximate CI for the mean ... but you'd need to account for the error in the estimate of the three parameters.

  2. However, you don't need to know the distribution to form a confidence interval for the mean since you could (for example) use a bootstrap confidence interval. With a reasonable-sized sample of positive values, that should work pretty well.

  3. In addition, there's also the possibility of using a normal approximation. The sample size isn't quite large enough to make the result symmetric -- if I take the idea in 1. and match the mean and variance of the positive values you show to get a gamma and then sample a zero-inflated gamma with the same sample size you have, the sample means that result are still mildly skew, but it's not too bad:

    simulation distribution of sample means from 0-inflated gamma shows mild skewness

    I'd think that a confidence interval based on a normal approximation with sample size of about 6000 or more should have something close to the right coverage, and the total width should be about right.

    One nice thing about using this approximation is that sample size calculations are standard, and straightforward. These can be used to get the right ballpark for the sample size even when you're using methods in 1. or 2. (as long as it's not substantially smaller than the sample size used here, because otherwise the approximation may no longer be accurate enough).

|cite|improve this answer
$\endgroup$
  • $\begingroup$ thank you so much for your answer - it is very informative. I do have one question though - how you estimate the ~6000? $\endgroup$ – RDizzl3 Aug 30 '16 at 18:25
  • 1
    $\begingroup$ @RDizzl3 Simulations show that sampling a population with your sample size (with similar characteristics) produces a slightly skew result. To my eye it is just okay at that sample size, but you can go down a bit without impacting the shape much, so I just rounded down (if I try simulations it still looks okay to me). [Your criteria may differ from mine; indeed you might think 4500 is fine or you might really not be happy until 8000.] $\endgroup$ – Glen_b♦ Aug 30 '16 at 22:10

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.