I am working with data that has been reduced to values falling between -1 and 1. The formula is, "(number in tube 1- number in tube 2)/ (the sum of numbers in both tubes). Example: Tube 1 = 5 worms, Tube 2 = 3 worms... ((5-3)/(5+3))= 0.25. Positive values indicate attraction, negative values indicate repulsion, 0 indicates no preference. In my experiments I put 40 worms in, and check to see which worms are in which tubes at different times (30 minutes, 1 hour, and 2 hours). How can I calculate if a single value, 0.25 for example, is significantly different than 0? I'll need to do this for each time I check the experiments. If someone could please help, I'd be grateful. Also, I use R, so if someone could demonstrate for me in R code, that would be swell. Thank you.
-
$\begingroup$ I'll need to do this for each time I check the experiments. Can you explain why ? Why you do not want a single test ? Do you stop the experiment once significativity is reached ? $\endgroup$– brumarCommented Jul 4, 2015 at 9:11
-
$\begingroup$ I apologize. I did want a single test. I just wasn't thinking correctly. Thank you very much for the help $\endgroup$– Spartan117Commented Jul 4, 2015 at 14:50
-
$\begingroup$ No problem, this idea struck me after giving you an answer, it's all my fault. My answer is still valid, but maybe a bit under powered depending on the worm behavior. I don't know if I can improve it based on these details but I would like to know why you want to check multiple times : it won't be stable after the two hours ? If so, it would be interesting to know how your worms migrate across time. Do they migrate a lot or not a lot across time ? My answer works well if they migrate a lot but I can adjust it. $\endgroup$– brumarCommented Jul 4, 2015 at 15:12
-
$\begingroup$ The issue with the time checks is that the response of the worms increases with time. At the beginning of the experiment I put 40 worms inside the holding tube connected to the two response tubes (Tube 1 and 2). As an example I observe that 3, 14, and 17 worms have moved into Tube 1 at 30 minutes, 1 hr, and 2 hrs, respectively. Tube 2 has 5, 16, and 20 worms at 30 minutes, 1 hr, and 2 hrs, respectively. Once the worms go in the response tubes, they cannot get out, so each subsequent time measure builds upon the last. In my example above, 11 more worms traveled to Tube 1 between the 30 minute $\endgroup$– Spartan117Commented Jul 4, 2015 at 15:52
-
$\begingroup$ time check and the 1 hour time check. In addition, by the end of the experiment (the two hour time check), the response is not always 100%, as illustrated above. How does all of this factor in to multiple comparisons, taking into account that the number of worms responding will likely be different for time points within a replicate and also the same time points between replicates, even though I always place 40 worms in at the beginning? Thank you again, by the way. $\endgroup$– Spartan117Commented Jul 4, 2015 at 15:58
1 Answer
Wait until the end of the experiment and compute the p value of the binomial distribution. Under the null hypothesis, a worm has equal chances to go to each tube. $X$ which represents the number of counts in a tube, follows a Binomial Distribution with $p=0.5$ and $N$ N representing the number of worms that have moved. Let's say 40 worms have moved, 13 in the least populated tube. The test is given by :
> binom.test(13,40,0.5,alternative="two.sided")
Exact binomial test
data: 13 and 40
number of successes = 13, number of trials = 40, p-value = 0.03848
alternative hypothesis: true probability of success is not equal to 0.5
95 percent confidence interval:
0.1857290 0.4912949
sample estimates:
probability of success
0.325
This is all you need to assess significativity of the relative attractivity of your tubes. The next section explains how to find critical values for your score if you still want use it as a statistic.
You will remark that your score is not used there but this is not impossible, for each $N$, to find critical values for which you will get significant results. If you want to do that
What you have to do, then, is to compute the critical region for which the null hypothesis $H0:P=0.5$ is rejected. That is to say $m1$ and $m2$ for which under the null hypothesis: $P(X \leq m1)=0.025$ and $P(X \geq m2)=0.025$ with . Let's say say that 50 worms have been used, and only $N=40$ worms have moved.
We can find $m2=27$ and $m1=13$. This can be given by using the quantile function of the binomial distribution qbinom.
> qbinom(0.025,40,0.5)
[1] 14
> qbinom(0.975,40,0.5)
[1] 26
You need to get one count further to enter the rejection region you want.
Indeed :
> pbinom(14,40,0.5)
[1] 0.04034523
> pbinom(13,40,0.5)
[1] 0.01923865
This can be read as $P(X \leq 13)=0.019$. By symmetry you also have $P(X \geq 27)=0.019$
This means that under the null hypothesis if you have 13 or less worm in one tube or the other, you are in the 5% most extreme cases.
If we get back to your score for $N=40$, when your score is superior in absolute value to to (27-13)/40=14/40=0.35 then you can say you have a significant attraction or repulsion.
Edit : Not really relevant anymore in light of Spartan details. The best seems to be to do one unique test at the end of the experiment.
However, beware at multiple testing. If you check this frequently you will increase your false positive rate. If you plan to check this 3 times, the simplest way to deal with it is to divide you $\alpha$ level by 3 and then recompute your critical regions with 0.00833 instead of 0.025. This would give :
> qbinom(0.008333,40,0.5)
[1] 13
>pbinom(12,40,0.5)
[1] 0.008294502
Same for the other guy, m2, which is pushed just one count further. The new threshold for your measure is then (28-12)/40=16/40=0.40.
Generally I would advise against multiple test if you can avoid it (like just checking one time). But one the other hand your critical region for the three check condition is kind of nice, 0.00829 is very close than the 0.0083 level. It implies that your test has not the problem of conservativeness that you had for the only one-check condition (your real alpha level was around 0.04 instead of 0.05).
