Should I interpret based on descriptive or inferential statistics in the present case?

Question

In our study which is completed, I have three groups: Drug A (n=87), Drug B (=62) and Drug C (n=5). The Shapiro-Wilk test, histograms and side-by-side box plots were used to assess normality of the data of the individual groups with respect to all the parameters used separately (weight, BMI, blood sugar, total cholesterol, triglycerides, LDL-cholesterol, HDL-cholesterol, clinical / illness / adverse effect / quality of life, etcc, scores).

So, ultimately we check for non-normal distribution and outliers, and are concerned about unequal sample sizes (n=5 in the Drug C group), in addition to looking at descriptive statistics (mean ± SD and proportion of % wherever applicable), nonparametric tests were carried out for the inference: For between-group comparisons, Kruskal Wallis H and subsequently for the therapeutic effect where p<0.05 obtained in Kruskal Wallis H, Mann Whitney U test as post-hoc test were used. And in case of within-group comparisons, Freidman’s test (for five evaluable visits data) and subsequently for p<0.05 for the therapeutic effect where p<0.05 was obtained in Freidman’s test, Wilcoxon-signed rank test as post-hoc test were done and p<0.05 was obtained using Wilcoxon-signed rank test in two (out of three) groups. Also, to ascertain the between-group significance difference, Mann Whitney U test was applied.

Kindly advise me:

1. First of all, have I made the right choice of tests and am I going in right direction?
2. With regards to the present analytical sample of N=154 in total, for arriving at final conclusion should I depend on descriptive or inferential statistics or both?
3. In the case where the mean ± SD changes and proportion of percentage (%) are found comparatively higher in case of Drug C which has sample size n=5 compared to other two drug groups but in case of between-group and/or within group inferential statistical analysis, (wherever) found non-significant difference / change in case of Drug C, can it be ‘said’ superior to other two drugs ‘on the basis of descriptive statistics’ or can I at least make a statement that mean ± SD and proportion of % changes at endpoint (or at other time-point) compared to baseline were found higher in case of Drug C compared to other two or other one? So, what should be the (cautious) interpretation / statement in case of Drug C compared to other two drugs?

Answer 1

It is good practice to check modeling assumptions such as goodness of fit for normality. Outlier rejection without external evidence would be bad practice. But outlier detection is a good descriptive technique. If formal tests for normality reject normality and the sample size is not very large and descriptive plots like qq plots also indicate non-normality then using nonparametric tests to compare treatment groups is appropriate. Since you have 3 groups Friedman's test and Kruskal-Wallis would probably be appropriate. Pairwise comparison of groups could be done using the Wilcoxon rank sum test (but p-values should be adjusted for multiplicity). This assumes normality is rejected and you want to reach conclusions about differences between groups. The Wilcoxon signed rank test is only used for paired data. Since you don't have that here I do not see how you could have applied it appropriately unless there is some way to pair on subjects within groups for within group comparisons.

If the data looks reasonably close to normal the parametric ANOVA F test could be used with pairwise comparisons using t tests and multiplicity adjustment.

Exploratory analysis is important initially to help understand the data and prepare to do inference (if inference is the goal). Regarding your question about descriptive vs inference the answer depends on your objectives. It appears that you want to make conclusions about the difference between the three treatment groups. In that case the descriptive statistics should be used to help formulate the appropriate hypothesis test. Ultimately, if inference is your objective you need to do the formal tests. Conclusions about differences between treatment group C and the other groups has to be made using the formal tests. if the sample size is too small (group C n=5) to reject the null hypothesis that group C differs from the others then you can't use the descriptive results to claim a difference. Mean and standard deviation estimates for each group can be presented for descriptive information.

Some of the methods and terminology I mentioned may not be clear to you. If that is the case ask questions and I will try to answer them in comments.