If you want to check how the scores of a given individual evolve from second 1 to 100 then you are dealing with a time series. A common time series modelling approach is the ARIMA models.
However, I'm not sure they are suited to your purposes. These models are intended to model trends, business cycles or seasonal patterns and to perform short-term forecasting. They are commonly used with variables such as gross domestic product, temperatures,... They can be also used to identify outliers or points that depart from the mean but I think that the methods mentioned below would be more appropriate in this case.
If you want to analyse how all the evaluations for a particular second or scene are distributed, then you have cross-section data (a variable observed across different individuals). Here, I would start with a histogram or a box-plot of the scores given by people at a particular second. Then you could perform a linear regression of those scores as a function of variables such as gender, age or other variables that you may have about the individuals.
You can make a temporal interpretation. For example you can display and compare box-plots of the scores at consecutive seconds. This would reveal the overall pattern of scores throughout time as well as the outliers identified in the box-plots.
Another idea that comes to my mind is to analyse the duration of batches of high or low scores. It is likely that a high score is followed by another high score and vice versa. (I assume that the scenes at each second are somehow related and there is a storyline behind them.) You could redefine your data for each individual as the duration (number of consecutive seconds) of good/bad assessments. For example, if the scores are over 7 you could define that second or scene as "good", if the score is below 4 as "bad" and in other case as "indifferent". Then you could analyse the duration of each state. This kind of analysis is known as survival analysis. Maybe you already known these methods, if not, you may start looking at Poisson regression. These models allow you also to include regressor variables such as gender and age. Thus, you could test if the duration of positive/negative perceptions depends on those characteristics of the individuals.
There are many textbooks where you can study the methods that I mentioned. Just for completeness I give you this reference.
The variety of statistical methods and models is huge and I just mentioned some approaches I am more familiar with. Let's see if some else gives more ideas.
