0
$\begingroup$

I want to compare how my computation model performs to the experimental data. For the experimental data I have proportions of different type of cells like the below table -

Cell Type Proportion
Type A 90%
Type B 1%
Type C 0.2%
... ...
... ...
... ...

For the model data, instead of single proportion values for each type I have multiple proportion values corresponding to the multiple runs of the model. Example of my model data, is the following -

Cell Type Proportion
Type A [90%,89%,92% ... ]
Type B [1%,0.8%,0.9%,1.2% ..]
Type C [0.2%,0.7%,0.11%.. ]
... ...
... ...
... ...

I want to test whether the values generated by model is similar to the experimental data. I am a bit confused about what statistical test to use. The following are my thoughts on this -

  1. Initially I was thinking of applying a simple t-test for each of the Cell Type. But I think , it is not recommended to compare t-tests for proportions data. And if I use that since I will be performing multiple t-tests, should I do some correction (Bon-Ferroni) to my significance level.
  2. Do I need to worry about the skewness of the proportion values (some are really high like 89% and some are really low like 0.2%).
  3. Can I use bootstrapping to get 95% confidence interval for each of the Cell Type, and then see whether my experimental proportion falls into the confidence interval as a test.

Kindly let me know, what statistical test to be used.

The other thing that is confusing me even more is the following - Since I want to test that my model is similar to the experimental data, hence my alternate hypothesis should be that my model data and the experimental data should be similar, but in all the above tests I am assuming this to be my null hypothesis and hence making it easier for me to pass these tests. This is because a p-value greater than the level of significance , will show that model is similar to experimental data.

$\endgroup$
3
  • 1
    $\begingroup$ (1) Counts instead of percentages are required. (2) Please repair the broken table. $\endgroup$
    – BruceET
    May 21, 2022 at 9:54
  • $\begingroup$ @BruceET Unfortunately I do not have counts. I have now repaired the broken table. Thanks $\endgroup$
    – Pranav Rai
    Jun 10, 2022 at 13:36
  • 1
    $\begingroup$ A formal test requires counts, not percentages. Ordinarily, you would need to have counts in order to do a chi-squared test of the null hypothesis that several categories are equally likely. For your data it seems that one could deduce from the uneven proportions what the counts might have been. $\endgroup$
    – BruceET
    Jun 12, 2022 at 0:04

1 Answer 1

0
$\begingroup$

Suppose there is a Type D to represent the $\dots$ at the bottom of your list. Then you might ask whether outcomes might be evenly apportioned among A, B, C, and D.

If there are $n = 1000$ altogether. You have (roughly) proportions $ c(.9, .01, .03, .06),$ which add to $1 = 100\%.$ Then, if you really did have $n=1000$ outcomes altogether, there would be strong evidence that categories are not equally likely.

    chisq.test(c(900,  10,  30,  60))

            Chi-squared test for given probabilities

data:  c(900, 10, 30, 60)
X-squared = 2258.4, df = 3, p-value < 2.2e-16

[Note: if no probabilities are given, the 'given probabilities' are taken to be equal.]

Even if you had only 90 outcomes, you could still get integer counts, as follows, and a highly significant result. (I don't see you your proportions could have arisen with fewer than 90 outcomes.)

chisq.test(c(90,1,3,6))

    Chi-squared test for given probabilities.

data:  c(90, 1, 3, 6)
X-squared = 225.84, df = 3, p-value < 2.2e-16

Notice that the chi-squared test requires counts and a journal editor for a paper on your work would very likely want you to give them. When counts are not given, one can only speculate what counts might have been as I have done above.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.