To keep or not to keep ... block effect when not significant?

Question

The question is in the title. I could not find an answer to this common question after searching on google nor on the StackExchange website, excepted this but the provided answer is not definite enough IMO.

The model is of the following form :

response ~ block + UnorderedCategoricalFactorOfInterest + error

It is not possible to test block:factor interaction because of no repetition in each block:factor cell, so I proceeded immediately to a type 2 test, in order to test main effects of block and factor.

It happens that block effect is not significant, and factor effect is highly significant.

From that point, what is best practice ? to keep it as is, or to re-fit the model without block ?

Answer 1

You should analyze according to the design, which was blocked. So, keep the blocks in the model.

Doing otherwise, that is, removing blocks from the model, inflated the degrees of freedom, and tests do not keep its frequentist properties. See When is it justified to "peek" at the outcome variable in model-building process?

If the experiment was randomized within blocks, a justification for this answer based on randomization theory (randomization tests) is that for testing, you should use the randomization used in the experiment.