1
$\begingroup$

I have this data generated from simulations. For each "prob" I observe the mean value of cases with "rule" = 1,2,3 or 4. So in the example below, for "prob=.05" all cases with "rule=2" have a mean value of "cc=.333..".

enter image description here
I cannot store each case since it would output a gigant dataset, and that is why I keep the mean values (among other summary statistics like the max values, min values standard deviation, number of cases, etc).

I am looking for a statistical test that tells me if the observed mean values are different for each rule, and that can deal with the fact that:

  1. these are non-normally distributed
  2. there are different number of cases per rule (30% are rule 1, 20% are rule 2, 10% are rule 3, etc)
  3. these are actually mean values.
  4. each rule has different standard deviations and standard errors

Any ideas?

Thank You in advance.

Addendum: I have different network topologies generated by the parameter "prob". At the end of each run, nodes end up being of one "rule". I would like to statistically know if node measures as the clustering coefficient (here "cc") or the eigenvector centrality measures (here "ec") are different for each rule as "prob" changes. When "prob=0" each node has the same values of "cc" and "ec", since the network is regular with same degree for each node. Indeed the node measures have a finite range since they are constrained by the network topology.

I do not think I can assume normality, if I look at the density plots of interest.

enter image description here

Improve this question
$\endgroup$
2
  • $\begingroup$ There might be more facts to exploit: 1) Your data seem to have a finite range. Also, you seem to get many (independent) individuals for each rule, so you can assume normality for the means. 2) If you can afford to compute also $\sum X_i^2$, you can exceptionally estimate the variances for each rule and prob by the non-stable formula $\sum X_i^2 - (\sum X_i)^2$. 3) Rules als prob can be considered as crossed factors, right? $\endgroup$
    – Horst Grünbusch
    Commented Feb 23, 2015 at 11:01
  • $\begingroup$ I already have measures for the variance of each rule, these are also outputs from the simulations. All rules can be observed for each "prob", as I explain in the addendum above. Thanks a lot. $\endgroup$
    – DianeM
    Commented Feb 23, 2015 at 11:45

2 Answers 2

Reset to default
1
$\begingroup$

If your sample is large enough and you suppose that for each pair of "rules" and "prob" the data is i.i.d., the Central Limit Theorem states that your sample means $\bar{X}_1,\bar{X}_2 \dots \bar{X}_k$ are Normally distributed with means $\mu_1, \mu_2, \dots \mu_k$ and variances $\sigma_1^2/n_1, \sigma_2^2/n_2,\dots \sigma_k^2/n_k$, respectively. Then, you can perform pairwise Welch's t-tests for difference in means

$$ \frac{(\bar{X}_i-\bar{X}_j) - (\mu_i- \mu_j)}{\sqrt{S_i^2/n_i+ S_j^2/n_j}}$$

As this is a pairwise test, you may compute, for a given "rule", the overall mean taking into account all "probs", except the one to which you want to compare to, and perform the test.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I have followed your suggestion of t.tests with the welch correction. In this case p.values are not as small as for the kruskal test which actually looks better. I still feel a little unsure about fulfilling the assumptions. For instance, are my rules actually independent from each other? Since it is in a network context and given the actual algorithm for differentiation, the fact that a node has x number of rule 3 neighbours will definitely condition the rule of the node itself. Thanks a lot. $\endgroup$
    – DianeM
    Commented Feb 23, 2015 at 13:28
0
$\begingroup$

The appropriate test is the kruskal-wallis test for the case when all individual cases are available. In R:

    kruskal.test(data$ec[which(data$prob==.05)],data$rule[which(data$prob==.05)],data=data[which(data$prob==.05),])

output: 

Kruskal-Wallis rank sum test

data:  data$ec[which(data$prob == 0.05)] and data$rule[which(data$prob == 0.05)]
Kruskal-Wallis chi-squared = 497.1976, df = 3, p-value < 2.2e-16
Improve this answer
$\endgroup$
2
  • $\begingroup$ This answer works for the case where I do have all individual cases, still don't know how would I have to proceed if the only available information were the summary statistics. In the future, simulations will be bigger and I cannot afford to store all individual cases. $\endgroup$
    – DianeM
    Commented Feb 23, 2015 at 10:23
  • $\begingroup$ Additionally, for variables in which mean values are exactly the same for each rule, sometimes I get NaN in for the kruskal test (which is ok and expected) and for other variables which are the same for each rule I get p.values of 0 which is not correct since the distributions are equal. $\endgroup$
    – DianeM
    Commented Feb 23, 2015 at 11:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.