2
$\begingroup$

We have 5000 vehicles of different classes (trucks, small cars, large cars) with 100 sensors in each car measuring fuel consumption, distance traveled, average speed etc for some time period $t$ that can vary between each data post. At random time-deltas, all 100 sensor values are sent to a data center.

I want to build an anomaly detection algorithm to calculate the confidence we have that each post received has fields within reasonable values.

After doing some research, I have concluded that a good solution to model this type of data would be to derive the multivariate Gaussian distribution from a vector of all 100 sensors as well as a feature for the class of the car (small car, truck, etc).

The reason I believe this to be good is because some of the sensor data will be highly correlated which is exactly what I want to capture.

Has anyone solved a similar problem like this? Is anyone aware of better alternatives?

Improve this question
$\endgroup$
1
$\begingroup$

I think it depends on your definition of anomaly. Do you:

  1. want to find one "strange" observation?
  2. want to find one "strange" sensor?
  3. want to find one "strange" car?

For the first question a fixed effect model would be good, where the fixed effect comes from each sensor.

For the second question a fixed effect model would be good, where the fixed effect comes from each car.

For the third question your approach would be best.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Not sure i understand. There are only information about sensors in each observation not car. What i want is to find faulty observations. Could you plz expand your answere? $\endgroup$ – user3139545 May 28 '16 at 21:06
  • $\begingroup$ I thought taht you have: Several time series of observations for all sensors. Most cars have several sensors, right? So all of the three options above are possible. So do you just want to see option 1. if a specific observation is "strange"? $\endgroup$ – Otto_K May 29 '16 at 8:55
  • $\begingroup$ Yes. Only if this observation is strange in some way. $\endgroup$ – user3139545 May 29 '16 at 9:04
  • $\begingroup$ Also the distribution of cars is not gaussian. There is a over representation of large cars wich might make thinge worse? $\endgroup$ – user3139545 May 29 '16 at 9:05
  • $\begingroup$ This problem will not occur with Mixed-Models. Have a look at: en.wikipedia.org/wiki/Mixed_model $\endgroup$ – Otto_K May 29 '16 at 11:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.