analyse peak distribution in time series

Question

Sorry if my question is too simple, I don't have much of a background in statistics. So I'll just try to describe the problem I need to solve in practice.

I'd like at least to find out what known problem this can be reduced to with proper terminology, and possibly how to approach it in R.

I have many client profiles which include simple logs:

Simplified examples from 2 clients (consecutive days in the X and orders in the Y):

Client 1:

 5  5  4  5  6  5 
-- -- -- -- -- --

Client 2:

10  9  1  0  0 10 
-- -- -- -- -- --

Both datasets (6 consecutive days) sum up to 30 orders. However, the first is uniformly distributed, while the second has two high peaks in specific days.

My data contains years of such logs, for hundreds of clients.

The type of analysis I need is: given a client and a time frame, what's their behaviour? Do they do regular purchases (1st type above), or do they make big orders less frequently (snd type above)? If it is the second type, what is the frequency of the peaks? is it regular or random?

EDIT: I am not interested in doing this visually, for a few cases - if not for exploring which technique to use. I need to do this in a bulk fashion, having a few simple indicators as a result

Answer 1

The kind of analysis you need is called "longitudinal analysis" on "panel data". The time series deals with a behavior of one or maybe a few customers across the time, while you have hundreds of customers.

I think you should read the first chapter of "Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence" by Judith D. Singer and John B. Willett, a popular text on the subject.

You also mentioned "peaks". From the scarce description of yours, it appears that you may need "event studies", which is also covered in the text above -- read the first chapter in the second part of the book.

The standard techniques for panel analysis are mixed effect models and hierarchical linear models. These are available in most stat packages such as SAS, Stata or R.

Time series techniques are very powerful when you focus on the time aspect more than anything, and the number of customers is low. In this case you could apply vector autoregression (VAR), seamingly unrelated regressions (SUR) and the host of other techniques. However, I'm afraid that this may not work for you, because you don't seem to have the grouping for customers yet. Usually, you can't apply time series for hundreds of customers separately, so you have to group them somehow, aggregate the data. You can't usually apply time series methods on hundreds of customers also because the covariance and coefficient matrices become too big, and there's not enough data to estimate them robustly.

So, at the very least you need to apply data exploration techniques on panel data, and this topic is covered in the text I referred. Next, you may identify a few distinct groups, then apply time series to them separately or together, when the number of groups is manageable. Alternatively, you may decide to stick to longitudinal analysis.

Answer 2

Yes, the analyses which you have asked for, can be done in R.

The technique is called time series analysis. So, I would answer your questions, by taking them one after one:

1. given a client and a time frame, what's their behaviour? Yes, the behaviour of a client, with respect to time, can be analyzed in R; by converting the data into a time series data and doing the relevant exploratory analyses like exploring trends, seasonalities, outlier behaviour, etc.

2. Do they do regular purchases (1st type above), or do they make big orders less frequently (2nd type above)? Plotting the above time series data would help you answer questions like these. If you want to compare clients, then overlapping both their time series curves would help you understand their behaviour with respect to each other.

3. If it is the second type, what is the frequency of the peaks? is it regular or random? Yes, one can write functions like these for getting the frequencies of time series data, and as you are talking about peaks, I would also suggest you to do an outlier analysis too.

Finally, knowledge of auto-correlation, and thus about the ACF, PCF curves for identifying noise would also be very helpful.