0
$\begingroup$

First of al, thank you for your time.

Question

  • My independent variable is the amount of computations that are performed.
  • My dependent variable is the fit of the model (fitness)
  • My control variables are the tuning settings of the genetic algorithm

For each of the 3 (independent variables) I do 10 repeated measurements where I initialize the algorithm at a random point. I hope this clarifies my approach. I don' t know how to approach random computer initialization as 'subjects': so I can't determine if it needs a within-subjects or between subjects ANOVA to prove the statistical relevance. What would independent repeated measurements be categorized (as between or within subjects)?

Background:

I had to optimize a Genetic algorithm and one of the plots include a trade-off between computational effort and convergence (see: https://imgur.com/1oAAJwA) For 3 given settings I performed 10 independent runs, where each outputed a certain fitness [0-1]. Now I want to compute the statistical relevance of the (black) trend. My questions is: how would you classify these kind of independent computer measurements (Between subjects or Within subjects)? I am using the book "Discovering Statistics Using SPSS by Andy Field" to define the required statistical tests, though this is where I get stuck. Normallity is met (One-sample Kolmogorov-Smirnov test), and sphericity clearly not: so it is iether a Friedman's ANOVA or a Kruskal-Wallis test according to this book. Although seeing that I don't really use human subjects but independent computer runs I am not sure if this is the way to go. note: the variances in the figure are huge, primarily because of limited sampling size: though a bigger one was not required for the course I'm doing. The 95% confidence intervals have been cut of at a fitness of 1, because they cannot occur beyond 1 (result of small sample size) )

$\endgroup$
4
  • $\begingroup$ What exactly do you want to do, estimate a regression like $c = \beta_0 +\beta_1 f$ where $c$ is computing time and $f$ is fitness ? and you have observations $(c_{ij}, f_{ij}, i, j)$, where $i$ is the category, $j$ the run in each category and $c_{ij}$ measured computing time and $f_{ij}$ measured fitness for category $i$ and run $j$? $\endgroup$
    – user83346
    Nov 18 '15 at 17:13
  • $\begingroup$ For the time being I am trying to determine if the decrease in convergence with less computing power is statistically relevant (the black line in the figure I uploaded to imgur.com). I am confused in what kind of a statistical test I should use to do so (since I use all independent runs I do not know whether this would be Within- or Between Subjects). I read here that what is called as 'repeated measures' is similar to 'within', though would this be applicable? $\endgroup$
    – Simon
    Nov 18 '15 at 17:40
  • $\begingroup$ Well I think that it has to do with repeated measures, and you have to estimate the regression that I specified, then you can perform a statictical test whether the coefficient $\beta_1$ is different from zero, if it is, then the computing time does depend on the fitness, you see what I mean ? Do you know R ? $\endgroup$
    – user83346
    Nov 18 '15 at 18:07
  • $\begingroup$ Thank you for your time so far! There is no need to determine if the fitness is dependent on the computing time. I only intend to show that the trend in black is relevant (without quantifying the trend). Therefore I was assuming an ANOVA test, to test the hypotesis that the means of the 3 catagories (3 levels of independent variables) are not equal. This would indicate some level of certanty for the trend at hand. Unfortunately the type of ANOVA depends on wheter you define it as within or between subjects. $\endgroup$
    – Simon
    Nov 18 '15 at 20:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.