2
$\begingroup$

Sorry if my question is too simple, I don't have much of a background in statistics. So I'll just try to describe the problem I need to solve in practice.

I'd like at least to find out what known problem this can be reduced to with proper terminology, and possibly how to approach it in R.

I have many client profiles which include simple logs:

Simplified examples from 2 clients (consecutive days in the X and orders in the Y):

Client 1:

 5  5  4  5  6  5 
-- -- -- -- -- --

Client 2:

10  9  1  0  0 10 
-- -- -- -- -- --

Both datasets (6 consecutive days) sum up to 30 orders. However, the first is uniformly distributed, while the second has two high peaks in specific days.

My data contains years of such logs, for hundreds of clients.

The type of analysis I need is: given a client and a time frame, what's their behaviour? Do they do regular purchases (1st type above), or do they make big orders less frequently (snd type above)? If it is the second type, what is the frequency of the peaks? is it regular or random?

EDIT: I am not interested in doing this visually, for a few cases - if not for exploring which technique to use. I need to do this in a bulk fashion, having a few simple indicators as a result

$\endgroup$
  • $\begingroup$ Because these clearly are not histograms, readers are likely to be confused by your question. I think the most common term for such data would be time series. It isn't evident that you are interested in peaks, for if you were you ought to characterize both of these series as having two peaks (in the sense of local maxima). Your final paragraph seems to be the clearest statement of the problem you are actually interested in. If that interpretation is correct, then please consider editing your post to emphasize that and to remove the potentially confusing language. $\endgroup$ – whuber Aug 24 '15 at 13:48
  • $\begingroup$ thanks, edited. I'm not only interested in peaks, the first case with no peaks is also part of the data $\endgroup$ – cornuz Aug 24 '15 at 13:55
  • $\begingroup$ Yes, that is something you can easily do in R. Both in terms of visualisation and analysis. It would be useful to see your data as a time-series to spot if there are peaks, how often the peaks happen and then investigate the mean of your data and the standard deviation (histogram would also be helpful here). Then you can dig deeper and see if clients behaviour change during weekends, etc.... $\endgroup$ – AntoniosK Aug 24 '15 at 13:55
3
$\begingroup$

The kind of analysis you need is called "longitudinal analysis" on "panel data". The time series deals with a behavior of one or maybe a few customers across the time, while you have hundreds of customers.

I think you should read the first chapter of "Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence" by Judith D. Singer and John B. Willett, a popular text on the subject.

You also mentioned "peaks". From the scarce description of yours, it appears that you may need "event studies", which is also covered in the text above -- read the first chapter in the second part of the book.

The standard techniques for panel analysis are mixed effect models and hierarchical linear models. These are available in most stat packages such as SAS, Stata or R.

Time series techniques are very powerful when you focus on the time aspect more than anything, and the number of customers is low. In this case you could apply vector autoregression (VAR), seamingly unrelated regressions (SUR) and the host of other techniques. However, I'm afraid that this may not work for you, because you don't seem to have the grouping for customers yet. Usually, you can't apply time series for hundreds of customers separately, so you have to group them somehow, aggregate the data. You can't usually apply time series methods on hundreds of customers also because the covariance and coefficient matrices become too big, and there's not enough data to estimate them robustly.

So, at the very least you need to apply data exploration techniques on panel data, and this topic is covered in the text I referred. Next, you may identify a few distinct groups, then apply time series to them separately or together, when the number of groups is manageable. Alternatively, you may decide to stick to longitudinal analysis.

$\endgroup$
  • 2
    $\begingroup$ @cornuz, this answer is the one you are searching for. Will vouch for it. Anyways, I would let my answer also stay, just in case you might use it later in the analysis. $\endgroup$ – Dawny33 Aug 24 '15 at 14:33
  • $\begingroup$ Thanks. As I was mentioning, understanding how to describe my problem properly helps a lot already. I fear it wont' be easy to translate this into working examples, not being familiar with the topic, but I'll do my best. Should anyone feel the need to come up with actual R code snippets, please don't hesitate ;) $\endgroup$ – cornuz Aug 24 '15 at 14:45
  • 1
    $\begingroup$ @cornuz, the long road starts with a first step. You probably need just a couple of days reading to start getting an idea of how this is all applied. $\endgroup$ – Aksakal Aug 24 '15 at 14:49
2
$\begingroup$

Yes, the analyses which you have asked for, can be done in R.

The technique is called time series analysis. So, I would answer your questions, by taking them one after one:

1. given a client and a time frame, what's their behaviour? Yes, the behaviour of a client, with respect to time, can be analyzed in R; by converting the data into a time series data and doing the relevant exploratory analyses like exploring trends, seasonalities, outlier behaviour, etc.

2. Do they do regular purchases (1st type above), or do they make big orders less frequently (2nd type above)? Plotting the above time series data would help you answer questions like these. If you want to compare clients, then overlapping both their time series curves would help you understand their behaviour with respect to each other.

3. If it is the second type, what is the frequency of the peaks? is it regular or random? Yes, one can write functions like these for getting the frequencies of time series data, and as you are talking about peaks, I would also suggest you to do an outlier analysis too.

Finally, knowledge of auto-correlation, and thus about the ACF, PCF curves for identifying noise would also be very helpful.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.