How to report robust anovas (WRS2::t2way and t3way): no df

Question

I have done experiments with a parasitic wasp species, comparing its life table parameters (response variables: longevity (in days), number of offspring and development time of offspring (in days)) when offered two different hosts (treatment: host 1 vs. host 2). The sample size on each host was 18-20 (some died) and the experiment was repeated four times (with a sample size of 5 each time). Since the sample size was quite small (and due to losses uneven) the resulting data are not normally distributed and heteroscedastic. To be on the safe side I used robust methods (t2way and t3way of the R package WRS2) for the analyses, e.g.:

t3way(longevity ~ treatment * sex parasitoid * run, tr = 0.2)

t2way(no of offspring ~ treatment * run, tr = 0.2)

I reported the results with the values provided in the R output (effects, and p-values for the effects). Since the robust methods are hardly ever used in my field of research, they are not known and I was asked to deliver degrees of freedom, or explanations why I cannot provide them. The t2way R-help states that no degrees of freedom are reported since an adjusted critical value is used, but I do not understand the meaning of that.

My questions are:

How do I properly report the results from both the t2way and the t3way?

And how do I best explain the method?

Answer 1

You may need to do some further digging. From the little digging I've done (e.g., https://www.sciencedirect.com/topics/mathematics/square-regression-estimator) , this is what I think is going on:

Each of the two (robust) anova procedures you performed reports p-values for some underlying tests. For example, the 2way procedure will report a p-value for a two-way interaction between two factors (among other things).

For the sake of argument, we can focus on the test of the null hypothesis Ho: there is no interaction between the two factors versus Ha: there is an interaction.

To perform this test, one can derive the sampling distribution of the test statistic assuming the null hypothesis is true. If the observed test statistic obtained in the current study exceeds a critical value (i.e., a 1-alpha quantile of the sampling distribution of the test statistic under the null hypothesis), then we can reject the null in favour of the alternative.

The sampling distribution might be a chi distribution with known degrees of freedom, etc.

Through simulations, it was found that the above critical value can be "adjusted" in order to improve the performance of the test. The adjusted critical value is simply the old critical value plus some complicated quantity.

In other words, instead of rejecting the null hypothesis when the observed test statistic exceeds the 1-alpha quantile of the null distribution of the test statistic, we now reject the null hypothesis when the observed test statistic exceeds a quantile plus some adjustment.

It seems that the p-value itself is found by trying out several values of alpha for the above procedure and identifying the one with desired properties. As per the link I shared above:

"Note that a p-value is readily computed using a pinch process. That is, determine the smallest level α that yields a significant result" (based on the equation of the test statistic)".

I still think one could report the name and degrees of freedom for the distribution used to derive the 1-alpha quantile. Maybe you can find some of that information if you peek under the hood of those functions.