I am currently working on non intrusive water (consumption) monitoring. I got time series data of flow rates at a level of 0.5 Hz (1 measurement at every 2 seconds) for a time span of about half a year. Detecting single extractions like a toilet flush is quite simple. However, I also want to detect water consuming appliances like washing machines and dish washer.
Scroll down to see the question and skip the intro
Analyzing and classifying single water extractions does not work because they would be interpreted as toilet flush or tap. Therefore, I need to take the context of such extractions into consideration. I have ground truth data of several non consecutive days (e.g. 2 days in December, 2 days in February, ...). Based on that ground truth data I know how a program or sequence of extractions from a washing machine usually look like (see extractions marked red int the figure below). They all have a specific start and end patterns in common (extraction of 10 seconds, short break, extraction of 100 to 150 seconds).
I tried several approaches, to identify start or end patterns in order to detect a washing machine:
- Dynamic Time Warping comparing a reference pattern of a washing machine with the data. No success so far due to noise and intermediate extractions in real life, like a toilet flush.
- I tried Motif using the R-package TSMining. No success because window size is not fixed in data. Washing machines adjust their consumption based on load and program. I also thought about Motif using DTW as a kind of distance measure, but my computing power is far too low for the amount of data of let's say 20 days.
- Currently, I am working on some kind of finite state machine. Based on water extractions and their context I try to calculate, if a washing machine is likely or not.
As mentioned before, a big issue is noise. I tried several noise reduction algorithms, but it is impossible to remove it properly. As you can see on the figure below, noise can be about +/- 2 liters/minute.
I also had a look at FFT, but I did not really know how to apply it properly due to the noise.
How can I identify reoccurring sequences (e.g. of washing machines and dish washer) in my case? A more general approach would be preferred. So I could identify reoccurring patterns independent from specific appliances. Assigning labels to some kind of clusters of patterns would not be necessary for now.
(Keep in mind, based on household protocols about washing tasks, I could NOT identify a rythmic pattern (e.g. every 3 days at 18:00).)