4
$\begingroup$

I have a limited statistics background, so I will try to be as specific as possible.

I'm reading a paper on a method to fingerprint mobile devices using their accelerometer. To demonstrate that the method is robust, the author measures parameters from 16.000 devices; I don't know if it's relevant, but the parameters are the Offset (O) and the Sensibility (S) of the sensor.

He then filter them by keeping only the values of devices which has been measured exactly twice. For every device, he calculates the "distance" between every O, and then between every S. He finds out that

The 95th percentile for O distance is 0.045. For S this distance is 0.0037.

First question: what does that mean? Moreover, what is the 95th percentile, and why does he use it?

He then uses these 95th percentile distances to create a scatter plot of O-S, dividing it into blocks of equal size and counting the data points in each block (Cxy). From this he is able to calculate overall entropy of the distribution (which he does not define) as follows:

enter image description here

Second question: what does the entropy tells me? I know it's a measure of disorder in a system, so I assume that it says something like "there are no clusters of similar informations, all the informations are disordered in the graph". But, how much? Shouldn't I compare this value with something?

The paragraph is then concluded saying that

Small variations of the grid origin had minimal effects on the entropy estimate (specifically, we saw less than 0.01 bit of entropy difference between the smallest and largest estimated value, 7.493 vs. 7.502). We consider this to be a confirmation that the result is robust.

Sorry for the long quote, but I really can't figure out what does that mean: what does it tell me if entropy has a small variation between the smaller and the largest estimated value? That's the third question.

You can find the paper here: Paper

Improve this question
$\endgroup$
4
  • $\begingroup$ This question seems to be too broad. It also is confusing. Please make it clear what source(s) you are referring to? $\endgroup$
    – Michael R. Chernick
    Commented Jan 16, 2017 at 12:31
  • $\begingroup$ I'm sorry for it being confusing. I added a link to the paper at the bottom of the thread. $\endgroup$
    – Alessandro
    Commented Jan 16, 2017 at 12:43
  • 1
    $\begingroup$ The question seems clear enough. The first thing to note is that the authors are coming at their analysis from an information theoretic perspective. This field has developed since the original Shannon-Weaver papers and framework in the post-WWII era and, while it has overlap with statistics, is really a separate discipline. Next, entropy is an I-T measure of global uncertainty and, as such, is a different metric from one that would support their claim of "unique" identification of a mobile handset. Clearly given your confusion, it's not all that helpful. $\endgroup$
    – user78229
    Commented Jan 16, 2017 at 14:27
  • $\begingroup$ Finding a better metric is more of a statistical question concerning accuracy or rate of misclassification. Finally, using the 95th quantile (or whatever) is a purely subjective choice on the part of the authors of this paper. As with the statistical significance of p-values at the 0.05 level, it is, at best, a rule of thumb or convention with no theoretical motivation. $\endgroup$
    – user78229
    Commented Jan 16, 2017 at 14:29

1 Answer 1

Reset to default
4
$\begingroup$

First question: What does that mean? Moreover, what is the 95th percentile, and why does he use it?

The 95th percentile for O distance is 0.045. For Sz this distance is 0.0037.

The x-th percentile is the value which is higher than x% of all values in a dataset. Per definition, the median is the 50-percentile. The percentile can be used to described a distribution. One reason to use the 95th-percentile could be to disregard outliers - those with the highest 5% distance.

Second question: what does the entropy tell me?

Entropy describes a distribution and can be well visualized and understood as a property of a histogram.

The entropy is zero, if only a single event (here: distance value) occurs - therefore there is no uncertainty. The histogram has only a single bin. If all events occur with equal probability (or if working with empirical data: occurred equally often), the entropy is at its maximum.

As the entropy depends on the number of possible events, it can be normalized such as dividing by the maximum entropy possible. Doing so, the entropy can only be a value in the interval of [0,1].

In this specific paper, the data is plotted as a scatterplot and the scatterplot area is discretized in equally sized sub-areas to form a histogram by counting the occurrences in each area. An area is hereby defined as an event and the amount of occurrences inside are counted.

Third question: what does it tells me if entropy has a small variation between the smaller and the largest estimated value?

Now the center of the grid - which originated from using equally sized areas for all values below the 95th percentile - depends on the data (!). By varying the grid center (maximum and minimum as possible alternative centers) and checking the change of the entropy, the author tries to estimates the robustness of his approach. If a small change of the grid center would result in a large change of the entropy, then his approach would be sensitive to the input data.

I assume there are plenty other options to estimate the robustness. I specifically wonder why the author does not estimate robustness of the identification as opposed to the robustness of entropy depending on the grid center.

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.