1
$\begingroup$

I have several images taken in a location every few minutes. The images depict a flooding pattern in a location which we count as number of pixels (y) vs a number that corresponds to time intervals. We have several stacks of images, each with about 200-300 images that we graph as number of pixels vs. time. The graphs look like something in the first graph. As you see there are several points that are much further away from the main grouping of the other data (the 3 points on the bottom and the one on the top) enter image description here

I would like to have my program automatically tag these images that are far away from the main data without me having to look at the graph.The purpose is to visually inspect them to find what is wrong with the images. I have used a running mean (mean=mean(y-5,y+5)) to follow the data like in the second graph. enter image description here

Then I was thinking that I could use the standard deviation to find the images of interest. For example if data(1)-mean(1) > 3*standard deviation (for the top) or mean(1)-data(1) > 3*standard deviation)(for the bottom) then I tag this image to be inspected.

The question is about the standard deviation I am using. When i find a 'local' standard deviation (using the 10 neighbouring data as I used in the running mean) then I get a lot of images for re-inspection that are very close to the main line. It does not just give me those images that are visually very far away from the main band of points, it gives me many more images so it does not work as well. But, if instead of the local standard deviation i use one common standard deviation from all the data ( standardev_all=standardev(y) ) then I get very satisfactory results and I would like to use this method.

In order to use that though I would like to perform a test to prove that the variance is equal throughout the graph (because basically i use the mean from the closest neighbors and the standard deviation from all the data) and I was wondering if using a test for variances in a set of data like that is appropriate. Right now the method i am using works well empirically but i want to make sure it is sound statistically. I know that there are several tests (f-tests,Bartlett's, etc) but I was wondering if these data the way that are described are appropriate to use these tests.

$\endgroup$

1 Answer 1

1
$\begingroup$

You can have a look at Gaussian Processes for modelling time series, cf. this paper for a clear exposition of their use for time series. I think it is an appropriate statistical tool for your problem.

Basically, using Gaussian Processes you can obtain a (say 95%) credible bandwith around your observations, thus you can take actions on the points that are not in the bandwith.

$\endgroup$
1
  • $\begingroup$ i looked over the Gaussian processes and it includes training data and I would rather not go that route because I have hundreds of these stacks and having a simple solution with a running mean like I described works really well and it is very fast. I just would like to know if the assumption that I make that the variance is considered equal throughout the graph is sound statistically (and how to prove that.) $\endgroup$
    – AL B
    Nov 17, 2015 at 22:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.