Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I'm testing throttle position sensors (TPS) my business sells and I print the plot of voltage response to the throttle shaft's rotation. A TPS is a rotational sensor with $\approx$ 90° of range and the output is like a potentiometer with full open being 5V (or sensor's input value) and initial opening being some value between 0 and 0.5V. I built a test bench with a PIC32 controller to take a voltage measurement every 0.75° and the black line connects these measurements.

One of my products has a tendency to make localized, low amplitude variations away from (and under) the ideal line. This question is about my algorithm for quantifying these localized "dips"; what is a good name or description for the process of measuring the dips? (full explanation follows) In the below picture, the dip occurs at the left third of the plot and is a marginal case whether I would pass or fail this part:

Print out of a suspect part

So I built a dip detector (stackoverflow qa about the algorithm) to quantify my gut feeling. I initially thought I was measuring "area". This graph is based on the printout above and my attempt to explain the algorithm graphically. There is a dip lasting for 13 samples between 17 and 31:

Sampled data shown with the "dip" magnified

Test data goes in an array and I make another array for "rise" from one data point to the next, which I call $deltas$. I use a library to get the average and standard deviation for $deltas$.

Analyzing the $deltas$ array is represented in the graph below, where the slope is removed from the above graph. Originally, I thought of this as "normalizing" or "unitizing" the data as the x axis are equal steps and I'm now solely working with the rise between data points. When researching this question, I recalled this is the derivative, $\frac {dy}{dx}$ of the original data.

Analysis of the derivative...?

I walk through $deltas$ to find sequences where there are 5 or more adjacent negative values. The blue bars are a series of data points who are below the average of all $deltas$. The values of the blue bars are:

$0.7 + 1.2 + 1.3 + 1.4 + 1.8 + 2.5 + 2.9 + 3.0 + 2.5 + 2.0 + 1.5 + 1.0 + 1.2$

They sum to $23$, which represents the area (or the integral). My first thought is "I just integrated the derivative" which should mean I get back the original data, though I'm certain there's a term for this.

The green line is the average of these "below average values" found via dividing the area by the length of the dip:

$23 \div 13 = 1.77$

During the testing of 100+ parts, I came to decide that dips with my green line average less than $2.6$ are acceptable. Standard deviation calculated across the entire data set wasn't a strict enough test for these dips, as without enough total area, they still fell within the limit I established for good parts. I observationally chose standard deviation of $3.0$ to be the highest I would allow.

Setting a cutoff for standard deviation strict enough to fail this part would then be so strict as to fail parts which otherwise appear to have a great plot. I do also have a spike detector which fails the part if any $|deltas - avg| > avg+std dev$.

It's been almost 20 years since Calc 1, so please go easy on me, but this feels a lot like when a professor used calculus and the displacement equation to explain how in racing, a competitor with less acceleration who maintains higher corner speed can beat another competitor having greater acceleration to the next turn: going through the previous turn faster, the higher initial speed means the area under his velocity (displacement) is greater.

To translate that to my question, I feel like my green line would be like acceleration, the 2nd derivative of the original data.

I visited wikipedia to re-read the fundamentals of calculus and the definitions of derivative and integral, learned the proper term for adding up the area under a curve via discreet measurements as Numerical Integration. Much more googling on average of the integral and I'm lead to the topic of nonlinearity and digital signal processing. Averaging the integral seems to be a popular metric for quantifying data.

Is there a term for the Average of the Integral? ($1.77$, the green line)?
... or for the process of using it to evaluate data?

share|improve this question

migrated from math.stackexchange.com Nov 1 '13 at 11:32

This question came from our site for people studying math at any level and professionals in related fields.

    
I think "average dip" is good enough. It doesn't have the dimensions of acceleration, so it's certainly not anything to do with that. –  ShreevatsaR Oct 19 '13 at 3:49
    
And I would appreciate any observations or commentary about this topic as a whole. I am a bit disturbed at how this "gut feeling" measurement isn't better expressed mathematically. –  Chris K Oct 30 '13 at 6:21
    
Could you possibly add in all the data points that you used to construct the ideal line, or add in a little bit more information about how the dotted red line is computed to justify the blue bars being the "deltas who are below the average of all the data points"? If it is morally the average distance from the average, then there should be an acceleration-style name for it, replacing of course differentiation with taking an average. –  Glen Wheeler Oct 30 '13 at 21:30
1  
Migrated from Math.SE by OP request: meta.stats.stackexchange.com/questions/1845/… –  Willie Wong Nov 1 '13 at 11:34
1  
I might add the word "local" to make it clear that step 1 exists -- I agree with @Glen_b (another Glen -- hi!) that this is important. So I would tentatively suggest "local mean defect" where I just concatenated "deviation from the ideal" to "defect". Seems suitable. –  Glen Wheeler Nov 10 '13 at 4:43

1 Answer 1

up vote 3 down vote accepted

First of all, this is a great description of your project and of the problem. And I am big fan of your home-made measurement framework, which is super cool... so why on earth does it matter what you call "averaging the integrals"?

In case you are interested in some broader positioning of your work, what you would like to do is often referred to as Anomaly detection. In its simplest setting it involves comparing a value in a time-series against the standard deviation of the previous values. The rule is then if $$x[n] > \alpha SD(x[1:n-1]) => x[n]\text{ is outlier}$$ where $x[n]$ is the $n^{th}$ value in the series, $SD(x[1:n-1])$ is the standard deviation of all previous values between the $1^{st}$ and $(n-1)^{th}$ value, and $\alpha$ is some suitable parameter you pick, such as 1, or 2, depending on how sensitive you want the detector to be. You can of course adapt this formula to work only locally (on some interval of length $h$), $$x[n] > \alpha SD(x[n-h-1:n-1]) => x[n]\text{ is outlier}$$

If I understood correctly, you are looking for a way to automate the testing of your devices, that is, declare a device as good/faulty after it performed the entire test (drew the entire diagonal). In that case simply consider the above formulas as comparing $x[n]$ against the standard deviation of all values.

There are also other rules you might want to consider for the purpose of classifying a device as faulty:

  • if any deviation (delta) is greater than some multiple of the SD of all deltas
  • if the square sum of the deviations is larger than a certain threshold
  • if the ratio of the sum of the positive and negative deltas is not approximately equal (which might be useful if you prefer smaller errors in both directions rather than a strong bias in a single direction)

Of course you can find more rules and concatenate them using boolean logic, but I think you can get very far with the three above.

Last but not least, once you set it up, you will need to test the classifier (a classifier is a system/model mapping an input to a class, in your case the data of each device, to either "good", or "faulty"). Create a testing set by manually labelling the performance of each device. Then look into ROC, which basically tells you the offset between how many devices your system correctly picks up out of the returned, in relation to how many of the faulty devices it picks up.

share|improve this answer
    
I believe "why on earth it matters" is a function of your own username. :) Why? Same reason there is an iliac crest: we need words to distinctively quantify everything unique in life. Imho, this QA is an example of how limited the vocabulary is within statistics. We need to combine confusing or contradictory descriptors for what is "to the eye" so simple. –  Chris K Dec 28 '13 at 9:28
    
Hehe, well spotted Sir! :) If I omitted any ventures into the land of creative branding it was merely because I felt compelled to support the resourcefulness and dedication of your effort and ideas rather than to concoct vain labels. Since you insist on naming the mean of the integral, beware that what you consider the "mean of the integral" is a simple mean of your deltas. And as such, your outliers are simply "deviations from the mean", or possibly deviations from the local mean. I don't quite see the advantage of thinking in integrals, unless you don't have enough sampling points. –  means-to-meaning Jan 4 at 0:44

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.