# Testing the parameter of a sub-sample

I have some survival data from which I compute the annual failure rates. The parameter in which I am interested is the mean failure rate.

Now I select certain subjects that have a given characteristic, so the sub-sample is not random. For example, I have very few subjects having the characteristic in the beginning and I have more and more subjects with time. I want to "prove" that the change in the mean failure is not statistically significant.

To do so I use bootstrapping. I select randomly sub-samples from the data having the same number of subjects per year as the original sub-sample and compute the mean failure rate.

Then, I can compute the a p-value using the bootstrap distribution and eventually reject or not the null hypothesis.

I have some questions regarding this procedure:

• is it reasonable?
• what would be the alternatives?
• what is in this case (bootstrapping) the difference between hypothesis testing and confidence intervals?

## EDIT:

Let me give an example of type data I have.

A number of N subjects are followed over the years. The population is not constant, each year some new subjects can be added or some subjects can be removed. The subjects can be in 3 states {"healthy","ill","dead"}. Obviously, "dead" is an absorbing state, but recovery is possible from "ill"ness. I am interested in the mean failure rate per state ("healthy","ill").

I estimate annual failure rates per state with a cohort approach: I compute the number of subjects that are in states "healthy" and "ill" at the beginning of each year and look which ones have died at the end of the year. The estimator is imposed, I do not want to change it for now. But advices are welcome!

Now, assume that I have 2 types of subjects: presence of a certain gene or not. I am interested in the effect of this gene on the mean failure rates. Due to selection of subjects, it happens that there are few subjects with the gene at the beginning of the study and more and more subjects with the years.

I want to test if the mean failure rates of the subjects with the gene are the same as the global ones. I have used bootstrapping to compute a p-value. Assuming that I have $h_t^w$ healthy and $i_t^w$ ill subjects with the gene at year t, I sample with replacement from $h_t$ (healthy) and $i_t$ (ill) subjects (with or without the gene) $h_t^w$ and $i_t^w$ subjects respectively and look how many have died.

• I don't understand this sentence: "I have very few subjects having the characteristic in the beginning and I have more and more subjects with time." – Michael R. Chernick Jun 13 '12 at 16:46
• As I do not yet understand your question I can't answer parts 1 and 2. Regarding part 3, it is possible to do the hypothesis test by inverting a confidence intreval (i.e. reject H0 if and only if the hypothesized value is outside the interval). However, for hypothesis testing centering based on assuming the null hypothesis is better and was recommedned by Hall and Wilson in their paper Hall, P. and Wilson, S. (1991) Two guidelines for bootstrap hypothesis testing. Biometrics 47, 757-762. – Michael R. Chernick Jun 13 '12 at 16:50
• @MichaelChernick The sentence means that e.g. first year I have only 5 subjects having the characteristic (compared with 100 for the global data), second year 6,..., last year 68 (compared with 143) etc. I was thinking that I cannot simply take random samples, so that I need to sample with the same number of subjects as in the original sub-sample over the years, so that I have bootstrapped samples under the null... – teucer Jun 13 '12 at 21:11
• Are you talking about new subjects each year or the cumulative number over several years? – Michael R. Chernick Jun 13 '12 at 21:18
• @MichaelChernick new subjects each year. – teucer Jun 13 '12 at 21:47

## 1 Answer

I don't understand how you're bootstrapping, but you're probably doing it wrong. Bootstrapping would entail simulating observations conditional on the outcome process, the censoring processes, and the distribution of covariates, of which joint estimation is impossible. Additionally, you have correlated data. Subjects which "weave" in and out of the study cannot be observed during their censored times. You may alternately be aggregating data and using a log linear model approach in which bootstrapping might be a sensible approach, but I still don't know how that would account for repeat observations.

Let me suggest a completely different strategy--

Your inference sounds very much like a motivating description for a proportional hazards model. By adjusting for the gene expression you're interested in, you can compute and compare risk sets of individuals averaged over instantaneous time and estimate a hazard ratio, which is an approximation of the risk ratio between those with and without the gene. By looking at "risk sets" at failure times, you can eliminate censored individuals from the denominator and have true apples-to-apples comparisons of instantaneous "at-risk" populations.

How many repeat illnesses does an average individual in this sample have as a proportion of the total illnesses? If there are only few, you may consider only the first such failure, or entering all failures as independent observations. If there are many such failures, you would want to use a frailty model.