2
$\begingroup$

I need to do a power analysis of a group that is composed of three subgroups.

The measurements to sample are difference measurements between two dogs of the same breed rated side by side. Dogs are sampled independently. There are three sub-categories below each dog breed. The null hypothesis is no difference between breeds (the parent group) (d = 0). The alternative is that there is a difference > 0.

So when we collect data, we have three rows:

  • Breed sampled
  • Subcategory
  • Difference measurement (continuous real number between -1 and 1)

For a typical power analysis from a group with no known subgroups, I'd simply use `pwr.t.test(n, d, type="one.sample").

However, I wasn't clear if this is a reasonable thing to do if you are powering a group that has known subgroups, each distributed differently.

Do we simply think of each sample from the parent group as being identically distributed, even though the subgroup distributions are different (but, in aggregate, they form another distribution we can think of as identical)?

Or should we instead model the subgroups and simulate the power needed by replicating the sampling process?

Improve this question
$\endgroup$
8
  • $\begingroup$ It might be implied from your question (or I may have simply missed it), however it is a bit unclear to me: What do you plan to test exactly? $\endgroup$
    – J-J-J
    Commented Jan 28 at 18:34
  • 1
    $\begingroup$ A difference between what? Breeds? Subgroups? $\endgroup$
    – J-J-J
    Commented Jan 28 at 19:16
  • 1
    $\begingroup$ @J-J-J Measurements of the dog breeds. Breeds, the parent group. $\endgroup$
    – Estimate the estimators
    Commented Jan 28 at 19:17
  • 1
    $\begingroup$ My thinking is the subgroups do not matter. You can always identify segments or subgroups within a population, and their distributions will vary from the overall population distribution. But we still treat the population as-is. This would only matter if there's covariate shift between the populations, I think. $\endgroup$
    – Estimate the estimators
    Commented Jan 28 at 21:08
  • 1
    $\begingroup$ Thanks for the edit, it makes the question much clearer to me (I upvoted it earlier). Now, I'm expecting an answer too. One could wonder why not using the "subgroup" variable in the future test/model (in particular as the different distributions of the subgroups could bring additional info), but I guess you already determined that you're interested in using this variable in your model. Nevertheless, even in this case, I wonder if determining power by simulation could be more accurate, as you could generate the "overall distribution" from the combination of the subgroup distributions. $\endgroup$
    – J-J-J
    Commented Jan 29 at 10:50

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.