In my experiment (pre-clinical vaccine testing) I want to know what kind of statistical test to be used to compare between 9 groups of animals (72 animals randomly divided into 9 groups). Each group consists of 8 or 7 animals each. After administering different experimental vaccines (n=9 for 9 groups of animals), each animal is evaluated for a continuous response (log10Titer) on unequal interval for about 1 year (0--day of vaccination, 7, 14, 21, 28, days, 1 , 3, 6, 9, 12 months after vaccination). After one year the animals are assessed for protection status (result is Yes or No for each animal). So there are 72 observations (yes or no). I have used linear mixed model with Tukey's multiple comparison test to find out significant group differences for time series data.

  1. Now I want only to use yes or no data from each animal to find out which group is best. What kind of statistical significance tests need to be carried out?
  2. p value for multiple comparison of percentage of protection of each group (calculated from Number of animals protected / Total Number of animals).
  3. Confidence interval of percentage of protection in each group.
  4. To asses which treatment group is best based on combined protection data and continuous time series data of one year.

I have SAS 9.3 and can work in R also (R Studio). I searched Google and found people suggesting different methods such as PROC MULTTEST PROC GENMOD / LOGISTICS. Some suggest Fisher's exact test and the chi-squared test. But in my opinion logistic regression / generalized linear model requires more data than it is used here.

My data look like this:

data animal;
input   Animal No   Treatment   Protection;
3   T-01    0
53  T-01    0
58  T-01    0
59  T-01    0
66  T-01    0
8   T-02    1
23  T-02    0
40  T-02    1
44  T-02    1
49  T-02    1
55  T-02    1
57  T-02    1
11  T-03    0
18  T-03    1
20  T-03    0
32  T-03    1
41  T-03    1
43  T-03    1
67  T-03    1
74  T-03    1
19  T-04    1
21  T-04    1
22  T-04    1
24  T-04    1
38  T-04    0
45  T-04    1
51  T-04    0
69  T-04    0
10  T-05    1
30  T-05    1
31  T-05    1
47  T-05    1
50  T-05    1
56  T-05    1
70  T-05    1
72  T-05    1
2   T-06    1
4   T-06    0
6   T-06    0
9   T-06    1
15  T-06    0
48  T-06    0
64  T-06    0
79  T-06    0
5   T-07    1
7   T-07    1
14  T-07    0
28  T-07    1
33  T-07    1
37  T-07    1
68  T-07    1
12  T-08    0
16  T-08    1
27  T-08    0
36  T-08    0
39  T-08    0
42  T-08    1
60  T-08    0
1   T-09    0
25  T-09    1
26  T-09    0
52  T-09    1
54  T-09    1
63  T-09    1
71  T-09    1
75  T-09    0
  • $\begingroup$ The small number of animals in each group may make it difficult to find which vaccine is best if you use yes/no responses. Even if 8/8 are protected in a particular sample, the Clopper-Pearson 95% CI goes down to a fraction as low as 0.63 protected in the population; 95% CI based on 2/8 protected in a sample goes as high as a fraction of 0.65 protected. (See statpages.org/confint.html .) So it will be hard to distinguish among vaccines this way. A continuous measure like titer, perhaps integrated over the 12 months, might do better. $\endgroup$ – EdM Jan 3 '15 at 18:39
  • $\begingroup$ yes i agree with titer over 12 monthis also a indicator for vaccine efficacy. However, requirement by regulatory bodies is by challenge of animals. the result indicates yes/no each animal. so the correlation need to be calculated for protection and antibody titer versus each vaccine type, which is my 4th research question. moreover preclinical stuides are aimed to screen for best vaccine compared to one standard vaccine group(T-02&T-03) and absolute control (T-01) and are thats why carried out with 8 animals in each group. Work was desined with statisticians help. now i don't have that help. $\endgroup$ – R.P. Tamil Selvan Jan 7 '15 at 5:34

If you concentrate on the crucial pre-planned comparisons of protection (each test vaccine against no treatment and against known-effective vaccine), the correction for multiple comparisons is not such an issue as it is in post-hoc analyses of results discovered in the data. That will tell you whether each of the test vaccines is better than no vaccine, and if any are significantly different from known-effective vaccine. The small number of cases, however, means that you will not have much power for detecting true differences from known-effective. If you have two equivalent standard vaccines (T-02 and T-03), you might want to combine those into a single group to get greater power.

The Fisher exact test (fisher.test() in R) is your best choice for comparisons. You don't have to worry about whether the requirements for the chi-square test are met, and with these small numbers of cases you won't run into computer overflow errors. You should look at the Wikipedia page to start learning about the controversies over the conservative nature of the Fisher test (and other discrete tests) and for references. In practice, the conservativeness of the Fisher test should tend to balance the issues raised by multiple comparisons, although I don't know a way to gauge the balance precisely.

Looking for the "best" among the test vaccines is problematic. The nominally best, T-05, gave protection in 8 of 8 cases, but the lower 95% interval for it goes down to a fraction of 0.63 protected. (The binom.test() function in R provides 95% confidence intervals.) So any vaccine that gave protection in at least 5 of 8 cases (T-04, T-07, T-09) can't really be distinguished from it. Ideally, you would use these results to choose a set of best candidates and investigate them more extensively in a larger study.

Do not give up on using logistic regression to analyze the relation of protection to a continuous measure of titer. Unless you have reason to believe that the relation of titer to protection will differ among the vaccines, including all 66 animals (or at least the 61 who received any vaccine) together in such analysis will show if there is an overall relation between titer measurements and protection, and could support use of titer as a surrogate biomarker for protection in future studies. You might consider including vaccine as a random effect in your model to see if there is any evidence of differences among vaccines in titer-protection relations, although again there may be power limitations.

| cite | improve this answer | |
  • $\begingroup$ now i compared each of test vaccine with no vaccine and standard vaccine. each test and standard vaccine is unique as there is additional factor i.e primary or booster applied for differentiating T-02 and T-03, T-04 and T-05, T-06 and T-07, T-08 and T-09. all groups differ in this additional factor. T-01 is absolute control. this animals will always suffer due to disease as no vaccine is applied in this group. $\endgroup$ – R.P. Tamil Selvan Feb 9 '15 at 11:42
  • $\begingroup$ 2.i have seen some publications they used bayesian inference for calculating 95 % CI using uninformative beta(1,1). i am really not understanding this. ex: 15/16(protected/total), percentage of protection = 93.8% [95% confidence interval=71.1–98.5] estimated by bayes method. i tried in r using binom.bayes {binom} but it is giving something different values than what mentioned in publication. can any one help me how can i do that. $\endgroup$ – R.P. Tamil Selvan Feb 9 '15 at 11:49
  • $\begingroup$ 3. yes protection differ among the vaccine so i cannot correlate all titer data with protection. if i have to use vaccine as random effect is proc logistic or proc glimmix is correct for above data? i think it is not possible to use random statement in proc logistic. if it is proc glimmix with above data format i am not able to do proc glimmix. whether i have to change the data into group wise with group, protection and total as variables? some one can help me this regard. thanks $\endgroup$ – R.P. Tamil Selvan Feb 9 '15 at 11:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.