# Sampling Methods for Review of Many Hours of Surveillance Video Footage?

I'm curious is anyone has experience with or can point me to some resources for sampling methodology for reviewing surveillance video footage. For example, say we have 6480 hours of video footage (3 months by 30 days x 24 hours x 3 cameras) and we are interested in characterizing some activity that occurs (not necessarily enumerating). Say we know that individuals that engage in this activity tends to be present at night and more likely to be present during second month than the first or third months. The activity may happen multiple times throughout the monitoring period, so it isn't a matter of capturing it just once, nor are we interested in capturing every instance of the activity. We just want to sample enough of it to be able to characterize the population engaging in this activity. So by way of example, say the activity occurs 1000 times during the monitoring period (being more prevalent at night and during the middle of the monitoring period) and takes between 10 and 30 minutes per instance. The activity can overlap i.e. several individuals may, but don't necessarily have to, be engaged in the activity at the same time on screen. How do we sample this footage (say selecting 10 to 20 percent of the total footage) to be able to maximize the number of instances we observe while not wasting labor reviewing all the footage?

EDIT: As has been alluded to in some of the comments, there are actually two things going on here. The first is that I want to sample footage in such a way that I maximize the possibility of an individual being present on screen. Given that a person is present they may be engage in some activity, what I want to do is characterize the range of behaviors. For this I want to be able see enough individuals that I can say that I've seen representative selection of their behaviors. I have data on the first part of the problem. Without revealing too much, let me give you that actual data to play with. What follows are total counts of individuals seen on footage from a camera nearby the location I am interested in. I have summarized these counts first by hour and secondly by day of year. For example over five years, 1005 individuals were seen in hour 0 (midnight to 1:00 AM) during the three month monitoring window. Similarly, over five years, 29 individuals were seen on day 182 (at any hour). My idea is to use these counts to come up with weights for a weighted random sample. As a side note, if I were interested in estimating the total number of individuals that I would see if I viewed all of the footage I do realize I can use a Horvitz-Thompson estimator.

tab.24<-data.frame(hour<-seq(0,23, 1),
count<-c(1005, 851, 750, 562, 311, 176, 132,  83,  99,  93,  87,  83,
89,  82,  83, 114, 187, 148, 152, 199, 398, 767, 1002, 1100))
tab.day<-data.frame(doy<-seq(182,273),
count<-c(29, 19, 18, 31, 25, 24, 44, 49, 54, 42, 38, 75, 71, 62, 71, 96,
142, 157, 117, 77, 159, 107, 104, 126, 134, 91, 146, 192, 223, 181, 201,
210, 178, 128, 203, 204, 250, 176, 243, 196, 213, 187, 200, 167, 160, 141,
120, 162, 178, 171, 145, 103, 99, 75, 114, 148, 140, 81, 69, 77, 49, 59, 44,
54, 58, 51, 46, 47, 54, 49, 35, 35, 39, 46, 46, 60, 20, 22, 37, 18, 22, 28,
38, 29, 30, 17, 22, 10, 18, 9, 15,3))


First let's assume assume you truly want to maximize incidence of this activity in your sample. Let's also assume that the smallest clip we're willing to sample is fixed, let's say 1 hour for simplicity. In order to frame this in some probabilistic theory, it would be nice to be able to assume independence between the number of people doing this activity in any interval, which of course is not reasonable since if someone starts in the last couple minutes of one hour, they'll be doing it in the next. So, if you don't mind, I will approximate the problem by saying we want to capture as many people starting this activity as possible. Beside this, is the independence assumption reasonable in your case?

Moving on, with this set of assumptions, our only choice is, which hour of footage to watch next. Therefore the only relationship we can exploit is the relationship between time and the rate of this activity. So if we model this rate as a function of the hour, than hour of day, or day of week, or is a holiday, etc. are our only real features (unless you can extract features from the video like average brightness...)

For continuing this problem I would look into two things. First, The Poisson distribution which is the distribution used for modeling rates of independent events. Secondly, the multi armed bandit problem. Essentially, think of your footage as a slot machine, and each hour of day as a lever. The payoff is instances captured. When you sit down at the slot machine you can start with some beliefs (as it sounds like you do) in the form of a (probably gamma) prior of which lever is best, or no beliefs at all. But each time you play, you update your beliefs, informing your next draw.

Very interesting to think about sampling to solve such a problem ! I'd suggest stratified sampling, using 4 stratas :

1. Day, 1st and 3rd month
2. Day, 2nd month
3. Night, 1st and 3rd month
4. Night, 2nd month

About the allocations within each stratum, do you have an (even rough) idea of how many such activities occur within each stratum ? If you do, you can use the proportional allocation (which gives optimal precision on estimators withing each strata). For example, if you know that when you observe 100 activities, they divide up like this :

1. 5
2. 10
3. 35
4. 50

Then, if you want to sample 100 hours out of 4000, you can draw 4 simple random samples in each stratum like this :

1. 5 hours (out of 1000)
2. 10 hours (out of 1000)
3. 35 hours (out of 1000)
4. 50 hours (out of 1000)

Of course, you can define your strata in many other ways (or use more strata it matches the parameters of your problem better), but that's the general idea of stratified sampling.

EDIT : I did not mention unequal probability sampling on purpose. You could very well use probabilities proportional to the number of activities observed (say, per hour per day). This would give you an Horvitz-Thompson estimator of the total number of activities with minimum variance (0, in fact, if your probabilities are truly built upon the real number of observed activities).

However, if you try to estimate an Horvitz-Thompson of a variable that is weakly correlated with the number of activities, you could end up with a very bad precision. In your case, I understand you try to estimate a discrete variable describing characteristics of people engaging in the activity, and I take it you don't have prior knowledge on how these variables behave ?

This is why I proposed stratified sampling : it will perform reasonably well on the estimate of the total number of actitivies (and using the proportional allocation, you will observe a not-far-from maximum number of activities), and you don't risk ending up with very high variances on other estimators.

• Antoine. Stratified sampling is an important technique for trying to get a representative sample of a population. Bug the OPs question regards maximizing the number of incidents captured. This is the goal in his sampling. What does strata fixation achieve there? – jlimahaverford Mar 19 '15 at 10:49
• The way I understand the question, the goal seems to be to achieve minimum variance on various estimators of characteristics regarding people engaging in said activties, not truly maximize the number of activities observed in sampled footage. If the strata are wisely chosen, I think the total number of activities observed will not be too far from maximum given the sample size. – Antoine R Mar 19 '15 at 17:36
• I guess both of these objectives are explicitly stated, and they are really at odds with each other, if we assume that frequency and population characteristic correlate with time (in a non orthogonal way). – jlimahaverford Mar 19 '15 at 17:41
• I've been pursuing the idea of unequal probability sampling, but not necessarily stratified sampling. I'll edit my post shortly with some actual data and a clarification of the question. – Dalton Hance Mar 19 '15 at 17:55
• Unequal probability sampling is seldom used in my field (official household stats), mainly because it is "risky" (in terms of precision). See my edit. – Antoine R Mar 20 '15 at 9:38