2
$\begingroup$

In ecology, we are often working with very small samples, i.e. I only have 13 sites of each of the two farming practices I am comparing. Thus, I really don't want to discard any of my sites and thus I am planning to do weighting rather than matching. However, should I first try to match my sites first (without discarding any sites) and only do weighting if it is not possible to match my sites?

What is the difference between using matching weights and weights from a weighing analyses per se (eg using the package WeighIt)?

If I have do do weighting, would this lower my ability to infer causality considerably?

$\endgroup$

1 Answer 1

4
$\begingroup$

Weighting and matching are two methods that aim to do exactly the same thing, which is to create two groups that have similar distributions of covariates. Matching works by pruning: any units that are not matched are discarded, and the rest for the matched sample. Weighting works by assigning a weight to each unit, which increases or decreases its priority in the estimation of the effect. There are methods of matching that don't drop units and instead reorganize the units into strata, and matching weights are computed from the strata. In this blog post, I explain how matching weights are propensity score weights.

Both methods have the exact same causal property, because causality comes from the assumption that the right covariates have been balanced, and both methods seek to balance covariates. You can only make a valid causal claim if you have eliminated confounding, i.e., closed all backdoor paths. This assumption is orthogonal to the method you use to do so.

Both matching and weighting decrease your precision in order to reduce bias. With a small sample, imprecision dominates bias in the error of the estimate, so you should not use methods that decrease precision. If anything, I would recommend adjusting for variables using regression, which may leave some bias but will increase your precision.

That said, if you think you can make a generalizable statement about causality using a sample of 26 units, you should rethink your design. That is way too small to make any useful inferences, let alone a causal inference. I would recommend you either abandon this question, collect more data, or take a different methodological approach (e.g., using a qualitative analysis) to answering your question.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.