I have a question related to change detection. Application domain is robotics/planning.
There is a sensor detecting distance from obstacle (ultrasonic / sonar sensor) at a specific position
(x, y, theta) in the environment.
It returns some reading at regular time intervals. Lets say the reading is
R and over a period of time it records
+/- means variation due to sensor inaccuracies).
I introduce an additional object between the sensor and the obstacle at a distance
D < R) so that at the next instance
D is detected and returned
I remove the original obstacle and now the next obstacle is
D' > R) and at the next instance
D' is returned.
Is there a way to exactly (or with high probability) say that a changed occurred NOW (when I add or remove an obstacle)?
Most change analysis algorithms consider a run length before change point and some data after change point and indicate the position change occurred.
But none I have read so far say change happened NOW; even the "online" algorithms seem to need some burn in data.
I want to implement a method that takes the data vector and return if the latest data point was a change point.
Adding some sample data
Here is a sample set of readings from a trial run:
[4.246904919227158, 4.063425344645503, 3.8522606458184168, 4.089331294361679, 4.227116239146714, 4.1902677894197256, 4.2114944818819655, 3.8056165437493474, 3.856400573638567, 4.010168749304731, 3.9006359327215225, 4.228516948802346, 3.345646289458722, 3.9652605551178945, 3.887277610253342, 4.03333576199138, 4.080046765134659, 4.056694343861694, 4.071850586980991, 4.100334404631286, 3.9658145837839665, 4.123166010661199, 3.8648499221011803, 4.2663999562925925, 4.093156431199762, 4.030454419556623, 4.150180573287889, 4.036968026040318, 3.968487007085925, 4.0230405601135795, 3.8041071703789893, 3.969994970247766, 4.041273183800564, 3.9044735289368897, 3.9436795221011653, 4.31314266597137, 4.086383240385605, 4.058007914552306, 4.07536832934258, 3.992830928581128, 1.992831838099113]
As you can see, the first 40 values have a mean of 4 the last one has a mean of 2.
A possible Solution/hack
Since my work involves streaming data, this is the approach I am currently taking.
- Read a window of data (for now, my window size is 20 values) from the end of the stream.
- Run bcp (from R) on this window.
- Check for the posterior probability of the change at location 18. (for all the runs i just had, the last value is NA, hence ignore that, and the data is zero indexed, (calling R from Python using rpy2), hence, the position turns out 18 for window size of 20.
- Set a threshold of 70% for the posterior probability (for now in my experimental setting this works fine, I may have to work on getting a proper threshold later)
- If the posterior probability at location 18 > 70%, I return TRUE indicating the recent data point has a different mean, or "change detected", else return FALSE.
This may not be the most efficient way of doing it, but it is doing its job for now. I am using this approach to carry my work forward.
I will update the thread if I find a better approach.
Thanks you all for the help!