0
$\begingroup$

Suppose I am measuring 3 variables over time in 5 minute intervals. These variables are measuring the same thing, but using different metrics. My goal is to determine which one of these metrics is the best. When “unusual” behavior occurs one (or many) of the signals catches it, but the others don’t. Does there exist a statistical way to quantify how well one signal does compared to others based on the other’s signals inability to catch certain “unusual” behavior?

I could just do a count of the number of “unusual” events that are caught by each signal and choose the one with the greatest output, but what if my sample of time frame does not contain data that is representative of what might happen in the future?

As you can see, I am at a very rudimentary level of this problem and some suggestions/ideas would be greatly appreciated!

$\endgroup$
1
$\begingroup$

Generally, you need to operationalize the QUALITY of your measurement. Then you will have to systematically evaluate different scenarios. I will give an example.

First, you will need to define the quality of the measurement depending on the application such as 1) little noise and 2) sensitive to unusual behaviour.

Then you can use test signals to evaluate the answer of your metrics. As you did not provide any information about your measurements, I will use a simple example.

Lets say you want to measure temperature, and you have the choice of either a Resistance temperature detector, infrared sensors or thermometers. First, to estimate the influence of noise, you will try to create an experimental environment with constant temperature (impossible, but you can try to minimize any changes as far as it goes).

Then you can rank all three methods regarding their variance throughout a time series of 5 minutes.

Then you can create several scenarios to estimate the sensitivity of the measurement, where you will (in this example) use a heat source to identify the sensitivity of the measurment.

Again, you can use statistic evaluation such as the difference between the peak and the mean value as a measurement for sensitivy.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.