Comparing participant numbers in a systematic review I am doing a systematic literature review on theories for measuring user satisfaction. I have to rate the quality of these theories, and one of the criteria should be the number of participants in the initial validation. So, we assume that each published theory was tested at least once for the initial publication (not always true, but the ones who didn't bother get a zero rating in my evaluation), and the ones who used too little participants get a rating decrease. 
I don't have to be very precise in this. I will be rating overall quality on a five-point ordinal scale, and I think it will be sufficient to really have only three ratings of participants, "too little", "enough" and "a whole lot", and adjust the quality rating one step up for "a whole lot" and one step down for "too little". 
But there is still the tripping point: when do I set the cutoff point between the three categories? 
Note that I am dealing with very large variations in theory and experiment design here, and with generally bad reporting. I cannot create a cutoff based on common metrics for effect size such as Cohen's d, because most of these study don't report such a metric at all. 
Is there any way to create rough categrization criteria across studies based on just participant number and the complexity of the theory? For the number of variables, I think that dividing the number of participants by the number of variable relations proposed by the theory will be good enough for me. (So, if study A studies the relation of one variable to satisfaction, and study B studies the relation of three variables to satisfaction, study B needs three times as many participants to be considered the same quality). But what do I do from there? I thought of making a relative comparison by using quantiles from the distributions of all studies I find, and call, say, the worst 20% of studies bad and the best 20% good. But I am not sure this will be a good strategy, as I expect a skewed distribution, and software engineering empirical experiments are known to have lower standards than those in other disciplines. 
I am aware that I can't do a really good comparison here, as it requires data I don't have. But is there a way of generalizing based on number of participants which is not worse than useless? 
 A: There's no way to estimate what large or small Ns are unless you have some estimate of effects. So, the first thing you need to do is work out the range of plausible effect sizes. That will vary across models and model parameters. That's OK. The observed effects will also vary across studies with significant findings with the observed effects being higher in low N studies. You already recognize that findings of low N studies are less plausible so you should get a weighted average effect placing more value on the large N findings. Essentially, you need to first do a meta-analysis before critiquing the Ns.
After that you could use power analysis to work out cutoffs for adequate sample sizes. If you have a reasonable estimate of the effect then it's pretty simple. For example, if you have an effect size of 0.6 using t-tests then anything under 30 subjects per group is going to be less than 60% of finding the effect. With 45/group you get a power of 0.8 but you have to go all the way up to 60 to get 0.9. I'm not suggesting those levels per se but you should be able to come up with a reasonable argument to pick some values.
Even better would be to just report the post-hoc power of the studies using the aggregate effect size (or range of power from plausible range of effect sizes). This could be used as a way to generate a critique of the subject numbers in studies without having to generate some arbitrary cutoffs.
