4
$\begingroup$

I am currently working on non intrusive water (consumption) monitoring. I got time series data of flow rates at a level of 0.5 Hz (1 measurement at every 2 seconds) for a time span of about half a year. Detecting single extractions like a toilet flush is quite simple. However, I also want to detect water consuming appliances like washing machines and dish washer.

Scroll down to see the question and skip the intro

Available data

Analyzing and classifying single water extractions does not work because they would be interpreted as toilet flush or tap. Therefore, I need to take the context of such extractions into consideration. I have ground truth data of several non consecutive days (e.g. 2 days in December, 2 days in February, ...). Based on that ground truth data I know how a program or sequence of extractions from a washing machine usually look like (see extractions marked red int the figure below). They all have a specific start and end patterns in common (extraction of 10 seconds, short break, extraction of 100 to 150 seconds).

enter image description here

Tested approaches

I tried several approaches, to identify start or end patterns in order to detect a washing machine:

  • Dynamic Time Warping comparing a reference pattern of a washing machine with the data. No success so far due to noise and intermediate extractions in real life, like a toilet flush.
  • I tried Motif using the R-package TSMining. No success because window size is not fixed in data. Washing machines adjust their consumption based on load and program. I also thought about Motif using DTW as a kind of distance measure, but my computing power is far too low for the amount of data of let's say 20 days.
  • Currently, I am working on some kind of finite state machine. Based on water extractions and their context I try to calculate, if a washing machine is likely or not.

As mentioned before, a big issue is noise. I tried several noise reduction algorithms, but it is impossible to remove it properly. As you can see on the figure below, noise can be about +/- 2 liters/minute.

I also had a look at FFT, but I did not really know how to apply it properly due to the noise.

enter image description here

Question

How can I identify reoccurring sequences (e.g. of washing machines and dish washer) in my case? A more general approach would be preferred. So I could identify reoccurring patterns independent from specific appliances. Assigning labels to some kind of clusters of patterns would not be necessary for now.

(Keep in mind, based on household protocols about washing tasks, I could NOT identify a rythmic pattern (e.g. every 3 days at 18:00).)

Improve this question
$\endgroup$
7
  • $\begingroup$ Really great, thought-provoking question. In a sense though, you've put the cart before the horse in probing for advanced functional forms and techniques. The first question I would want answered is how many different types of patterns are you trying to identify? You've mentioned flushing toilets, washing machines and dish washers. Are there others? Showers? Dish washing machines vs hand washed? Etc. Then, do you have a "ground truth" in terms of a set of data where these various types of processes are identified and pre-classified? This would act as a training set. Start modeling with that. $\endgroup$
    – user78229
    Commented Mar 12, 2017 at 20:02
  • $\begingroup$ @DJohnson I tried to identify 1 pattern (the one above in the plot) which occurred in 2 households - it's a washing machine. There are of course other consumers like toilets, showers etc., but they only extract water once per use (in general). Finally, yes I have a "ground truth" set of labeled data. I used a random forrest to classify those fixtures which only extract water once (e.g. toilets), but classifying such extractions like the ones in the plot above ended in a disaster because I only have 5 samples of washing machines. Currently, I also have a look at density based clustering. $\endgroup$
    – Karl Regensburger
    Commented Mar 16, 2017 at 9:43
  • $\begingroup$ Have you ever built this type of model before? If not, why not start with the Fisher iris data classification example? It's a widely used data training set based on various metrics of, e.g., flower petal length, against which you can learn and apply several algortithms such as SVMs, linear discriminant analysis, clustering, and so on. $\endgroup$
    – user78229
    Commented Mar 16, 2017 at 10:41
  • $\begingroup$ @DJohnson yes, I built models with kNN and SVM with other data before. I also built them with my current data, but they perform worse than random forrest. $\endgroup$
    – Karl Regensburger
    Commented Mar 16, 2017 at 22:45
  • $\begingroup$ Have you tried an unsupervised, deep learning NN? $\endgroup$
    – user78229
    Commented Mar 16, 2017 at 22:47

1 Answer 1

Reset to default
2
$\begingroup$

This is similar to work identifying appliance use by monitoring electrical consumption patterns in time with one home smart meter, although they may have a little more information (e.g, phase for helping detect inductive loads as well as power draw), where you just have flow rate. They'll even have the same sorts of patterns over time for washing machines. This has been ongoing for a while, so I just did some quick Googling. These should provide help as a starting point for research in this area:
https://www.researchgate.net/publication/308764100 , https://cs.brown.edu/research/pubs/theses/masters/2011/mallya.pdf , and https://archive.uea.ac.uk/~ajb/Papers/LinesIDEAL2011.pdf
This would be easier for you if you could simultaneously access the electrical meter as well as water flows at the same time to have more features, but that's probably for a more distant future project!

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.