# What does it mean exactly, to “build a statistical model” of, say, a series of images?

I would like a meaningful and concise explanation for what it means exactly, when someone says, "We built a statistical model of all our images".

I overheard this, (and keep overhearing that phrase), but I am not sure how/what that entails exactly.

EDIT: Context:

The problem set here is as follows. From a UAV, video imagery of corn fields is collected en-masse. The problem is to be able to tell when a pest (usually a small-medium sized rodent/mammal) is moving. Usually they just took the difference between consecutive images. However in this case, many false alarms happened because leaves might sway in the wind, small changes occur that are not related to the rodent moving, etc. So to this end, they said "Instead of the difference between frames method, we instead build a robust statistical model of the images we have". So what does that mean exactly?

This is the context. Thank you.

• In essence it means to specify a model for the distribution of the observations (at least some of) whose parameters are estimated from data (the 'statistical' part). If there's no data, it might be called a 'probability model' instead, but it's not statistical. In the context of the problem you describe, there's certainly data (the images); some noise component would be specified for the observed images, so if you look for differences, many of the previous causes of false alarms would be seen by the model as part of that 'noise' and not trigger an alarm. – Glen_b Aug 28 '13 at 21:15
• @Glen_b "In essence it means to specify a model for the distribution of the observations", right, so thats what I am trying to figure out - does this mean they already used a canned PDF, or does this mean that they came up with their own PDF? This is what I am trying to understand. – Creatron Aug 28 '13 at 21:22
• There's no way to tell from the information I see here. It's not just the specific form of the distribution function, but the way the parameters of the observations relate that matters. – Glen_b Aug 28 '13 at 21:30
• @Glen_b Let me rephrase: If I use a known PDF to describe a set of data, then have I done 'statistical modelling'? In a different case, if I have invented an altogether new PDF to describe a set of data, have I also done 'statistical modelling'? I am trying to understand what, literally, that phrase means. – Creatron Aug 28 '13 at 21:34
• Yes in both cases, in the same way that if I dig my garden with a hoe I bought from a hardware store or one I fashioned myself in a shed, I'm still gardening both times. – Glen_b Aug 28 '13 at 21:35

I think they are modelling each pixel as a mixture of gaussians - this is a model of the background see eg opencv http://docs.opencv.org/modules/video/doc/motion_analysis_and_object_tracking.html BackgroundSubtractorMOG¶

An improved adaptive background mixture model for real-time tracking with shadow detection, Proc. 2nd European Workshop on Advanced Video-Based Surveillance Systems, 2001: http://personal.ee.surrey.ac.uk/Personal/R.Bowden/publications/avbs01/avbs01.pdf

My simplified understanding: you model each pixel (separately) as a MOG. Take 1 gaussian case- for each pixel you calculate running average, and standard deviation. So now you calculate z-score of intensity value wrt your gaussian model - if its an outlier you declare "rat"

so previously you were marking X[i,j,t+1]-X[i,j,t]>threshold ="rat"

[and more complicated with real mixture of gaussians]

To clarify- what is being built is a statistical model of a "background" pixel. The current method can be interpreted as assuming each pixel has constant intensity: taking difference (in time) identifies the foreground "objects". Next level up, we model background pixel values as normally distributed: we estimate the mean and standard deviation of each pixel by a running average calculation. This basically allows us to ignore "typical variations" - ie areas which are constantly changing pixel values are ignored (eg leaves on tree?) - you need a bigger change there than in areas of low "typical variation". Finally, Mixture of Gaussians allows you cope with more complicated "typical patterns of variation".

• Hi sean, I like this answer - can you please elaborate some more on your understanding here as it pertains to the example given? Thanks! – Creatron Aug 29 '13 at 13:59

The statement is somewhat ambiguous, because it does not specify what kind of model they built. But what they are saying is that they have found some PDF to describe the series of images. It might be using one of the techniques you describe, or it could be using something else like for example a restricted boltzman machine, just to pick one. Because images do not lend themselves very naturally to normal distributions, picking the kind of model used is a big part of building the model.

A bit more background if it isn't obvious: the standard first step is to take the image and represent it as a series of pixel values (0-1, 0-255, doesn't really matter), either as a 2-D matrix of pixel intensities, a 3-D matrix if it's color, or just flatten it to a vector.

What you do from there is the real trick. There is no standard statistical model for representing images, so if the author of the statement wants to be explicit they need to specify the model they used. So yes -- that's a very fair question. But conceptually having some PDF to describe the image set makes sense just and is meaningful regardless of what the underlying model is.

• I have edited the question to give you the exact context. Does that help? Thanks. – Creatron Aug 28 '13 at 21:11
• With context we can better speculate what they mean, but it is still just speculation. – Leopd Aug 29 '13 at 3:45