How to validate experimental data using statistics?

I am calculating shannon entropy from an experiment on Computer Network. The entropy is computaded from IP addresses in network. A have some entropy values in attack condition and another entropy values in "normal" condition. There is no standard values that I can compare. For example:

No attacks:

3.36247297985
3.41986105867
3.54255608391
3.40470906978
3.51915910324
3.53429621173
3.74549136098
3.44319360325


Under attacks:

1.80297042947
0.509454817684
0.529487864782
0.589599900586
1.41917987628
1.79816685786


What kind of "statistics formulas"/model I can use to validate the data/experiments?

Type of distribution the entropy : I dont know.

The size of data: there are about 50/60 values of entropy for attack/no attack in a text file

• In order to choose the correct statistical test can you give us some more specifics for the entropy data. When not under attack do you know what type of distribution the entropy values have? What is the size of your data for attack/no attack?
– Hugh
Mar 9, 2017 at 17:34
• @Hugh, type of distribution the entropy : I dont know. The size of data: there are about 50/60 values of entropy for attack/no attack in a text file
– Ed S
Mar 9, 2017 at 17:40
• Cross-posted: stats.stackexchange.com/q/266487/2921, math.stackexchange.com/q/2179541/14578, cs.stackexchange.com/q/71350/755. Please do not post the same question on multiple sites. Each community should have an honest shot at answering without anybody's time being wasted.
– D.W.
Mar 10, 2017 at 3:50

Since you don't know about the distribution of the entropy values you will have to use a non-parametric test.

Essentially you want to test if the populations are the same but "The populations are the same" is a very broad statement, usually we test a more specific hypothesis like "The medians are equal" or "The variances are equal". I know that your example values are just made up but for the example a test of medians would be very good. If the real non-attack entropy values are consistently near 3.5 and the attack values are much lower/higher then a median test is a good option.

A popular non-parametric test is the Mann-Whitney U test, it's easy to find software for it. Unfortunately this test can only be used when the attack and non-attack sample sizes are equal. This is still a good choice, you might consider randomly selecting values from your bigger data-set so that you get equal sample sizes.

An option for non-equal sample sizes is the median test.

If you have data for historical attacks then perhaps you could try both tests to see which gives fewer false positives/false negatives

• "I know that your example values are just made up" -> The values are real from my experiment! Thanks!
– Ed S
Mar 9, 2017 at 20:04