I haven't done much time-series modelling at all and now I have a dataset which screams "time series" at me and now I am hunting for a model.

The data: Imagine a video watched by ~400 people, 100 seconds long. Each person had a slider from 0 (bad) to 10 (good) to indicate if they liked a particular scene. The value of the slider is recorded each second, resulting in 100 variables from the first to the last second.

The problem: Now I would like to determine points of interest in this short film: e.g. seconds where the mean answer deviates from the overall mean of the film, indicating a good or bad scene. Is there some form of time-series model which can do this?

Thank you.


If you want to check how the scores of a given individual evolve from second 1 to 100 then you are dealing with a time series. A common time series modelling approach is the ARIMA models.

However, I'm not sure they are suited to your purposes. These models are intended to model trends, business cycles or seasonal patterns and to perform short-term forecasting. They are commonly used with variables such as gross domestic product, temperatures,... They can be also used to identify outliers or points that depart from the mean but I think that the methods mentioned below would be more appropriate in this case.

If you want to analyse how all the evaluations for a particular second or scene are distributed, then you have cross-section data (a variable observed across different individuals). Here, I would start with a histogram or a box-plot of the scores given by people at a particular second. Then you could perform a linear regression of those scores as a function of variables such as gender, age or other variables that you may have about the individuals.

You can make a temporal interpretation. For example you can display and compare box-plots of the scores at consecutive seconds. This would reveal the overall pattern of scores throughout time as well as the outliers identified in the box-plots.

Another idea that comes to my mind is to analyse the duration of batches of high or low scores. It is likely that a high score is followed by another high score and vice versa. (I assume that the scenes at each second are somehow related and there is a storyline behind them.) You could redefine your data for each individual as the duration (number of consecutive seconds) of good/bad assessments. For example, if the scores are over 7 you could define that second or scene as "good", if the score is below 4 as "bad" and in other case as "indifferent". Then you could analyse the duration of each state. This kind of analysis is known as survival analysis. Maybe you already known these methods, if not, you may start looking at Poisson regression. These models allow you also to include regressor variables such as gender and age. Thus, you could test if the duration of positive/negative perceptions depends on those characteristics of the individuals.

There are many textbooks where you can study the methods that I mentioned. Just for completeness I give you this reference.

The variety of statistical methods and models is huge and I just mentioned some approaches I am more familiar with. Let's see if some else gives more ideas.

  • $\begingroup$ WOW, I really liked your well thought out post - it's one of the best I've ever seen. This morning I got the final data - but the data are quite boring, basically: 0:00 is 50% then it increases to a 80% peak when some important information is shown and decreases to 70% and stays there. Not much going on there, especially no problematic etc. scenes are visible to the naked eye. I thinkt I will certainly do a Box-Plot per second, to see if some scene is more polarizing than the others, that is a very good idea. $\endgroup$ – Christian Sauer Jul 2 '14 at 6:24
  • $\begingroup$ Funnily, I wrote my master's thesis using cox regression, but I never thaught that I could use it on this dataset - the really like the idea, but the initial results are so clear cut, that it would probably be statistics for the sake of statistics. $\endgroup$ – Christian Sauer Jul 2 '14 at 6:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.