2
$\begingroup$

I'm trying to figure out how to analyse the data which consists of a number of visits to a website and the total amount the visitor ends up spending there.

There are obviously a lot of zeros - people leaving without buying anything - and few relatively large amounts, so the data is severely left skewed.

I'd like to transform it into some kind of normally distributed data, so I can analyse it properly, but I'm not sure how to go about doing that.

I've looked into various transformation functions, read about Weibull distributions, lognormal, chi-squared and goodness-of-fit, but for my lack of experience, it's too much information :-)

I'd like to get push in the right direction and maybe some comments from people who have worked on something similar.

Thanks!

$\endgroup$
4
  • $\begingroup$ can you plot your data? that may be of help. $\endgroup$
    – mugen
    Dec 29, 2014 at 10:46
  • $\begingroup$ I will try to get hold of actual data. The question was meant to be theoretical - perhaps it's easy to imagine how it would look in theory? $\endgroup$
    – jgivoni
    Dec 29, 2014 at 11:07
  • $\begingroup$ Do you have any substantive idea about why people spend how much they spend? Also zero inflation is definitely a good start $\endgroup$ Dec 29, 2014 at 15:57
  • $\begingroup$ I have absolutely no idea why people spend the amount the spend. One aim is to find variables that might predict expenditure. $\endgroup$
    – jgivoni
    Dec 29, 2014 at 16:08

2 Answers 2

4
$\begingroup$

The problem you face is called "zero inflated data." Most approaches to this are through either the negative binomial or zero-inflated Poisson distributions. Whether or not you can do a transformation depends on why you have zeros in the first place. There are discussions at http://www.theanalysisfactor.com/zero-inflated-poisson-models-for-count-outcomes/ and http://www.ualberta.ca/~bhumphre/class/zeros_v1.pdf. (Although the math is scary, read at least section 3.4 Discussion.)

$\endgroup$
1
  • $\begingroup$ Interesting, thanks. The zeros in this case is a choice and the decision to participate is probably sufficiently independent of the consumption decision so a Two Part Model appears to be the best fit. $\endgroup$
    – jgivoni
    Dec 30, 2014 at 9:43
2
$\begingroup$

You cannot transform data with a large number of 0's to normality.

You say you want to "analyze it properly" but you don't say what you want to find out. It seems likely that this variable is the dependent variable in some equation - you want to predict customer spending based on various traits of the customer or the site or something. In that case, the place to start is regression. OLS regression makes no assumptions about the dependent variable except that it is continuous - it makes assumptions about the error as represented by the residuals.

If the residuals are not normal (which seems likely) then there are various things to try. You could try quantile regression (this might be useful, in any case, even if the residuals are normal) or regression trees and their offshoots (forests and so on).

$\endgroup$
2
  • $\begingroup$ Ok, so say I want to find out with which confidence one segment of customers appears to be bigger spenders that another segment? Can I do that if the data is not normal? $\endgroup$
    – jgivoni
    Dec 29, 2014 at 15:47
  • 1
    $\begingroup$ Sure. 1) As I said, there is no requirement for the data to be normal, only the residuals. 2) If the residuals aren't normal, investigate the methods I mentioned above. $\endgroup$
    – Peter Flom
    Dec 30, 2014 at 13:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.