1
$\begingroup$

I have developed an Interactive Genetic Algorithm for my thesis and I need to find out if the mean population fitness at generation 5 is significantly different from the mean fitness at generation 10. I need to claim that there is no significant difference between generation 5 and 10, as a reason why I am stopping the session at generation 5 (to reduce user fatigue).

Here is the data I have from 12 different users:

  Mean fitness Gen 5  Mean Fitness Gen 10
---------------------------
User 1 | 0.74167      0.69167
User 2 | 0.55         0.7
User 3 | 0.7          0.69145
User 4 | 0.875        0.734
User 5 | 0.6834       0.45834
User 6 | 0.89167      0.8
User 7 | 0.7167       0.7167
User 8 | 0.7          0.8167
User 9 | 0.725        0.834
User 10| 0.6834       0.875
User 11| 0.65834      0.85
User 12| 0.634        0.7

Mean     0.713265   0.738988333

Would a paired sample t test be the best option in this case?

I did this:

Null Hypothesis = 0 Alternate Hypothesis != 0

alpha = 0.05 t stat = 0.661888985 p value = 0.260833417 critical value (2 tailed) = 2.20098516

I cannot reject the null hypothesis because t < critical value, thus there is no significant difference.

Is that the correct way to do it? My problem is that I have always been told that you should always aim to reject the null hypothesis and here I am 'happy' that it was not rejected.

$\endgroup$
  • $\begingroup$ It is always a slippery slope when 'interpreting' null effects. $\endgroup$ – HEITZ Feb 19 '16 at 0:31
  • $\begingroup$ Is "User 1" the same entity both times? Note that failure to reject the null does not imply the null is true. $\endgroup$ – Glen_b Feb 19 '16 at 0:37
  • $\begingroup$ User 1 modified the fitness for 10 generations. Here I am only showing the mean fitness at Generation 5 and at Generation 10. I need to prove that there is no statistical difference between Gen 5 and Gen 10. $\endgroup$ – Alex Feb 19 '16 at 16:13
0
$\begingroup$

From what I can see that P value seems to indicate that your alternative hypothesis is that the difference (gen10 - gen5) is >0 and not !=0 (i.e. a one-sided test as opposed to a two-sided).

But anyway, for this analysis it doesn't matter the difference isn't significant in either case.

I think it's wrong to say that you should aim to either reject or accept the null hypothesis. But sure in most experiments(citation needed) you would like to see that what you did had an impact, so then you would be happy if you can see a significant difference.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.