13 votes

Sample Size impact on Effect size

When your sample size is small, there will be more sampling error attached to your effect size estimates and hence more uncertainty/wider confidence intervals. In that sense, I wouldn't say the effect ...
Christian Geiser's user avatar
9 votes

Sample Size impact on Effect size

As long as the sample is random, the estimate of the effect size is unbiased even in small samples. However, other things being equal, it will be less precise. It's still your best guess, but the ...
Peter Flom's user avatar
  • 117k
5 votes

Determination of sample size vs. repeated measurements in experimental design

Repeated testing of one widget will allow you to determine the magnitude of measurement variation (noise) and, maybe, if there are time or test-dependent changes in the performance. Those things are ...
Michael Lew's user avatar
  • 14.9k
5 votes
Accepted

How to calculate sample size from hazard ratio?

The key reference here is Schoenfeld (1983), who gives the following formula: $$ \frac{(z_{1-\beta}+z_{1-\alpha})^2}{pA\times pB\times\text{log}(\text{HR})^2} $$ This will give you the number of ...
PBulls's user avatar
  • 3,658
5 votes

Sample Size impact on Effect size

All things being equal, the major difference is the variation in the effect as a function of sample size. A very simple simulation in R shows this as the case. Here I simulate two groups who have the ...
Shawn Hemelstrand's user avatar
4 votes

Sample Size impact on Effect size

The question is fully addressed by meta-analysis (see, e.g., Introduction to Meta‐Analysis by Borenstein et al.), which includes studying of how the effect size vary from one study to another. Indeed, ...
Roger V.'s user avatar
  • 3,843
3 votes

Seeking Papers with Significant Associations in Univariate but not in Multivariable Analysis Due to Sample Size

You are comparing two different things: Univariate analysis assesses marginal distribution, whereas multivariate analysis assesses conditional distribution. They answer different questions. Both ...
2 votes
Accepted

Calculate sample size in an experiment with interactions

This looks like a two-way ANOVA disguised as a three-way ANOVA. The critical things I see that you are testing are these two factors: The effect of stimulation. The effect of treatment. (Optional) ...
Shawn Hemelstrand's user avatar
2 votes

Are there coventions on reporting weighted sample sizes?

I assume you will report weighted averages or proportions. Usually weights are based on a reference population,eg, the United States or New York City. This basis for weights should be reported. ...
David Smith's user avatar
  • 1,178
2 votes

Small-sample binary logit and linear models - response to referees

I do not think the problem comes from the small sample size per. It rather comes from the fact that MPs are obviously almost present everywhere regardless from the value of explanatory variables, ...
CaroZ's user avatar
  • 615
2 votes

Small-sample binary logit and linear models - response to referees

There is no fix for this situation. Not only has the experiment used a minimum information (binary) response variable, but it does not have the needed sample size (96) just to estimate the intercept ...
Frank Harrell's user avatar
1 vote

Small-sample binary logit and linear models - response to referees

Can we still use a multi-factor logistic regression model? 2. The results of the single-factor logistic regression analysis suggest that the D-dimer concentration index has statistical significance (P=...
DrJerryTAO's user avatar
  • 1,454
1 vote

Small-sample binary logit and linear models - response to referees

The usual rule of thumb with logistic regression is that you can include one IV for every 10 cases in the less common case. For you, that's six people, so this rule would say "One IV at most"...
Peter Flom's user avatar
  • 117k
1 vote

Determination of sample size vs. repeated measurements in experimental design

I want to compare the performance of two versions of a widget using a 2-sample comparison test. H_0 = both versions perform the same, H_1 = widget.b performs worse than widget.a. Your hypotheses ...
DrJerryTAO's user avatar
  • 1,454
1 vote

Sample Size calculation - “subgroup hypothesis”, Guidance needed

Quite a few question here, I'll provide some thoughts. Powering for overall vs. subgroups Unless the operating characteristics line up through (very unlikely) chance, you'll have to choose between ...
PBulls's user avatar
  • 3,658
1 vote

superiority margin in sample size calculation

An appropriate method depends on the details of your planned analysis. You have to look carefully at the documentation of these calculators to see if they are fit for your purpose. The first link (...
J-J-J's user avatar
  • 3,852
1 vote

Record all influential factors even at high n?

This is a cool use-case of applied causal inference! To answer your question directly: If there is no confounding and you have sufficient samples, causal inference is definitely a lot easier and you ...
Scriddie's user avatar
  • 2,084
1 vote

Seeking Papers with Significant Associations in Univariate but not in Multivariable Analysis Due to Sample Size

It would be interesting to see a simulation meeting your criteria. Usually in practical settings, the more variables you have the more probable it is that you will find the relationship between ...

Only top scored, non community-wiki answers of a minimum length are eligible