Causality in Online Classification

I'm using an SVM for an online classification, i.e. datapoints are classified as they come in. Of course, the training has occurred earlier, offline, with an annotated dataset.

However, the system has causality, i.e. each incoming datapoint is dependent on previous datapoints (they come from a sensor after all), so ideally, this should be taken into account for the classification.

Any advice on how I can incorporate that into my classification method? For now, no history is taken into account. Thanks!

Edit: By 'causality' I mean that the class-to-be-predicted is dependent on the previous class, as the physical system cannot jump from one state(=class) to another randomly, but some 'events' must happen, and hopefully these 'events' are captured in the features of the previous datapoints. The usage of the previous result of the classifier is also an option, although it may lead into a very rapidly accumulating error.

• What about adding the previous (lagged) observation(s) to the current observation ? this may not be the most rigorous way to proceed, but this can bring information. – RUser4512 Nov 12 '15 at 9:43
• Not sure I understand. You mean add the previous observation to the current observation and feed their sum into the classifier? I suppose I'd have to normalise (average) as well, otherwise the features will get potentially unnatural values. – PeriRamm Nov 12 '15 at 9:57
• Not the sum. Just the values of the $k$ previous observations. so that, if $k=1$, $y_t=f(x_t,x_{t-1})$ instead of $y_t=f(x_t)$ – RUser4512 Nov 12 '15 at 10:00
• Nice idea. I'll also add $k$ as a feature as to differentiate between observations. Thanks! – PeriRamm Nov 12 '15 at 10:16
• Its hard to help if you don't give a clear description of the real world problem. But you could use hidden markov models. basically it sounds like you want to predict current class given current inputs and that previous state was X. in machine learning the term seems to be en.wikipedia.org/wiki/Maximum-entropy_Markov_model (basically replace svm with logistic regression to get probability – seanv507 Nov 12 '15 at 11:38

An idea was given by RUser4512, which can be proven useful:

• Instead of training the classifier with only one observation (datapoint) per classification result, also use previous observations.
• Add another feature, 'history' to distinguish between observations ($k$ in the above notation)
• When classifying in real-time, also use previous observations.

The above solution may make the training dataset more demanding (there must be consecutive chuncks of observations), but may be well worth it.