1
$\begingroup$

I'm working on a thesis and I'm using a bit statistics to process the data. I have a certain population and in one sample (N=250) I measure the time it takes to reach a goal. Then I change something in the population that delays that time. I take another sample (N=250) and compare those two. I use a T test to get the difference between the averages of the two samples. So I know how much the average delay is. Now I'd also like to know how much percentage of the second sample has a delay. Is it possible to calculate this? And how?

EDIT

We are simulating a swarm of robots. One of them, the searcher, has to find a target by using the information of the surrounding robots. We do this 250 times and record the time it takes for the searcher to get to the target. This is our reference data. Then we add an enemy who sabotages the searcher and delays the time. We again run this 250 times and record the data. We are sure that some times the searcher gets to the target before the enemy can sabotage it because of the algorithm.

Improve this question
$\endgroup$
5
  • 2
    $\begingroup$ The way you've described it, the second sample is contaminated by some delay, the first sample is not and to estimate the average delay (call it $\delta$), you take the difference of the means. If that's the setup, the answer to "Now I'd also like to know how much percentage of the second sample has a delay." is trivial - 100%. If you change it so that some proportion (say, $p$) of the times in the second sample are not delayed, then the two sample comparison of means does not estimate the average delay - it estimates $\delta(1-p)$. $\endgroup$
    – Macro
    Apr 6, 2013 at 19:04
  • $\begingroup$ Wait I don't get it. First of all, I've only had one little course in statistics in college so I don't know a lot of it. $\endgroup$
    – Kat
    Apr 6, 2013 at 19:06
  • 1
    $\begingroup$ I'm saying that if only some of the people in the second sample's times were delayed, then the two sample difference in means does not reflect the average time delay - to estimate the average delay you will need to know how many times were delayed. Do you have that information? If so, just calculate the percentage like Peter Flom says. If not, I don't think it's possible - the two sample difference in means only estimates $\delta(1-p)$ - the average delay multiplied by the proportion of times that were delayed - you have to fix either $p$ or $\delta$ to estimate the other. $\endgroup$
    – Macro
    Apr 6, 2013 at 19:10
  • $\begingroup$ Sorry I wasn't finished with the comment. Anyway it is so that some proportion of the times the second sample are not delayed yes. So the percentage is not 100%. That I am certain. But I can't know which ones because there are a whole lot of random characteristics that decide the outcome.(So my samples are not actually matched. Is this a problem?) You say that the outcome of my t test is actually δ(1−p). How do I get p then? $\endgroup$
    – Kat
    Apr 6, 2013 at 19:13
  • 2
    $\begingroup$ However, you could probably learn something by looking at a histogram of the second sample. For example, if the sample is roughly normally distributed, the delay is large relative to the variance, and a non-negligible proportion were subjected to the delay, then you should see a bimodal distribution. The relative heights of the two modes should give you a good idea of what proportion of the points were delayed. To actually estimate that proportion in a principled way, you'd probably need a mixture model. $\endgroup$
    – Macro
    Apr 6, 2013 at 19:24

1 Answer 1

Reset to default
1
$\begingroup$

You've gotten some good information from @Macro. Let me say a few general things, and hopefully something may be of help to you.

First of all, your response variable is "the time it takes to reach a goal". This is time to event data (also called survival, duration, failure time, etc.). There are two big issues with time to event data: the first is that you almost always have censoring. One example would be that your study stopped before everyone reached the goal, and so you only know that it took some people longer than some duration $t$ (where $t$ represents, e.g., the total elapsed time up until the end of your study). Do you have censoring? If so, you will need to use survival analysis methods, not the t-test. The second is that event time data are rarely adequately normal. Are your data sufficiently normal? Even if you don't have censoring, it may be preferable to use survival analysis, or to use a non-parametric approach (such as the Mann-Whitney U-test), or to transform your response data to achieve normality.

The other issue that sticks out for me is that you say that "it is so that some proportion of the times the second sample are not delayed". How do you know this? Is it because the distributions overlap? (That would not provide evidence for that proposition.) Based on your description and the conversation in the comments, I cannot see how you could know that only some of the times are delayed, but not others. I see this issue as an example of a common misconception. For instance, when hurricane Katrina hit the US Gulf coast, there was a lot of discussion in the media about global warming causing more severe storms, and whether global warming caused Katrina, or whether it had caused other storms, but not that one. Such discussions indicate a misunderstanding of distributions and what the data tell us. Typically, if we have two distributions with similar variances but different means, e.g., $\mathcal N(4, 3)~\&~\mathcal N(6, 3)$, it's best to understand that a constant (namely, $2$) has been added to every value in the first distribution to create the shifted distribution. Of course, the world is always more complicated than your theory, but this is nonetheless the best way to think about this situation. (Note that this would not necessarily be true if the variances differ.)

I believe your question is more related to the second set of issues, and that they are what lie behind some of @Macro's comments, but I suggest you take the first set of issues seriously as well in thinking about your project.

Improve this answer
$\endgroup$
8
  • $\begingroup$ So the more information you guys give me, the more I'm hesitating about what to do. I added the project in the question for more information about what my data represents. I don't think we have censoring. The time we record is the actual time the searcher needs to get to the target or in some cases (we hace different kinds of enemies) it is a time that resembles infinity. Until now we always removed the infinity times from the sample and ran the t-test without it. So are we doing good statistics with the t-test or should we look for something else? (Also, is infinity simply removing correct?) $\endgroup$
    – Kat
    Apr 8, 2013 at 9:14
  • $\begingroup$ And no, the data is not normally distributed. But we found papers that said that if the sample size is big enough we could use the t test anyway. If it is correct to use the t test of course. $\endgroup$
    – Kat
    Apr 8, 2013 at 9:51
  • $\begingroup$ @Kat, what does it mean that "in some cases... it is a time that resembles infinity"? Are there cases where the searcher never finds the target before you get tired of waiting? If so, and you remove those before analysis, you will end up with biased results. You should probably work with survival analysis, possibly with a statistical consultant. $\endgroup$
    – gung - Reinstate Monica
    Apr 8, 2013 at 13:07
  • $\begingroup$ That is correct. But in the cases where infinity occurs we look at the percentage of times with infinity and not so much at the mean comparison because we know that the times that are infinity in the first case would have been the longer samples in the first, non-delayed case. If this is what you mean with biased, than we are well aware of this fact. $\endgroup$
    – Kat
    Apr 8, 2013 at 13:37
  • $\begingroup$ I don't quite follow that, @Kat. If you are using a t-test, you are comparing means. You're right that with sufficient data, you don't need to worry as much about normality, & w/ N=500, you're probably fine, unless the problems are extreme, but you will still have no way to account for the 'infinities'. You would be best to use survival analysis, IMO. $\endgroup$
    – gung - Reinstate Monica
    Apr 8, 2013 at 14:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.