I am wondering if anyone would have a succinct way of describing the differences between the two. My understanding is that the Fisherian way is non-parametric and relies on the randomization test to conduct inferences while the Neymanian method utilizes a distribution is an appeal to the central limit theorem for a Normality based inference.
Is there a general rule of thumb of which is better and what the differences are?
In short, your intuition is correct.
Fisher's approach is based on randomization tests: using the randomization procedure of the treatment assignment, you assume a sharp null hypothesis of no effect and then compute the exact p-value, that is, the probability of seeing an effect as big as the one observed assuming no effect.
Neyman's approach is based on estimation. You will estimate the average treatment effect and a conservative variance (the Neyman-Variance, equivalent to the HC2 variance) and, by the CLT, you can make inferences using a normal approximation.
For more information about both you should check chapters 5 and 6 of Imbens and Rubin book.
It's important to notice that these methods are not "causal methods" strictly speaking, they are simply estimation and testing methods, and usually discussed in the context of experiments, where it's the validity of the experiment that warrants the causal meaning. That is, there's nothing causal about them per se, in the sense that whether your estimate is causal or not depends on your identification strategy. To learn more about identification, it's definitely worth studying causal graphs --- you can find references here.
On whether one is better than the other, there's no answer to that without context, it depends on your goal. For example, do you care about the sharp null hypothesis? In a lot of cases people don't, since the exact null is implausible, so they care more about estimation --- if that's your case, then randomization tests will not be very useful for your problem.
