Determine sample size for each cluster

Question

Suppose I have a large population and I want to test if installing a new special light bulb can reduce energy consumption. Since I have a large population, I'll have people that usually consume high and some that consume less energy on a daily basis; therefore I would like to know the impact of the new special light bulb depending on their previous behavior. So I want to cluster my population into groups based on their energy consumption. Suppose that I get K groups after clustering. How can I determine the minimum sample size for each of my K+1 groups (+1 because I'm counting with the group that doesn't have the new light bulb, i.e, control group) to ensure some statistical rigor (for example some confidence level, etc..). I've only encountered literature about stratified sampling and cluster sampling, but none of those are what I want to do. In stratified sampling, they divide the population into K groups and then take elements from all of the groups to make sure every group will have elements in the (only) sample. In clustering sampling, they divide the population into K groups and only consider some of those groups for the sample.

EDIT 1: I found about one-way ANOVA: it can determine sample sizes for multiple groups but assumes as a null hypothesis that all of the groups have equal means (which is not the case here). In this case, the null hypothesis would have to be something like "there's no changing on the mean (energy consumption) on any of the groups with the new special light bulb". Even if one-way ANOVA could be used, I would have to give as an input a guess/estimation of the means for each group after getting the new special light bulb?

EDIT 2: I'd like to thank StatsStudent for his answer, which is very detailed and informative. However, his answer is not what I'm actually looking for. To be more detailed, I have the data on the energy consumption of the whole population (without the new special light bulb). And after clustering, I know there are K different types of behavior (for example, K=3: low, medium, and high energy consumption behaviors among the population). I want to know if installing the light bulb will affect the energy consumption of any of the K groups. Also, one aspect that would also like to include is to have more than one special light bulb, for example, three different light bulbs: A, B, and C. Can I use Multi-Way ANOVA to determine the sample size needed? I've read that this test is an extension of ANOVA, where there is more than one category, but only one category is of interest. The other category/categories are things that need to be controlled for (blocking/nesting/random effects/etc.). If I'm interested in determining if there is a difference in energy consumption due to light bulbs (A, B, and C) while controlling for energy behavior (low, medium, high) (K=3:groups from clustering) can and should I use two-way ANOVA? This will result in $3 \times 3 = 9$ groups.

Answer 1

You essentially have a before and after experiment in which each participating household is provided the new bulb at a point in time and measurements are taken on energy usage before the "intervention" of the bulb is introduced and then additional measurements are taken for the same amount of time after the bulb is introduced. In this case, you'll want to treat each household as its own cluster really and perform a paired-sample t-test (there are more complex designs that aim at reducing bias, but this is a good starting point):

Under the paired sample $t$-test, each household is measured for say, 1 year. After 1 year, the bulb is introduced and for the next year energy consumption is measured. Denote the energy consumption of household $i$ before the bulb is installed in time-period 1 as $Y_{i,1}$ and the energy consumption of the household $i$ after the bulb is installed at time-period 2 as $Y_{i,2}$, for $i=1, 2, ..., n$, where $n$ is the total number of participating households.

Then you'll compute the difference between the two time periods for household $i$ as:

$Y_{i,diff}=Y_{i,1}-Y_{i,2}$.

You'll then compute the simple average of of these $n$ differences. Call this $\bar{Y}_{diff}$. In addition, you'll compute the variance as $SD_Y^2={{\sum_{i=1}^n(Y_{i,diff}-\bar{Y}_{diff})^2}\over{n-1}}$ and compute the standard deviation as $SD_Y=\sqrt{SD_Y^2}$.

You will then want to test the Null Hypothesis, $H_0$, that the mean difference between time periods is equal to zero vs. the Alternative Hypothesis, $H_1$, that the mean difference between period 2 and 1 is greater than 0 (you're expecting the bulb introduced in time period 2 will result in less energy than in time period 1). In other words, you'll want to test:

$H_0: \mu_{diff}=0$ vs $H_1: \mu_{diff}>0$.

To test this hypothesis, You'll compare the test statistic: $T={\bar{Y}_{diff}\over{SD_Y/\sqrt{n}}}$ with $n-1$ degrees of freedom to the standard $t$-distribution with $n-1$ degrees of freedom. To perform this test at the 99% confidence level for example, you'd compare the value of your test statistic $T$ with $t_{.01}$ with $n-1$ degrees of freedom. If $T >t_{.01, n-1}$, you'd reject the null hypothesis and conclude that there is sufficient evidence at the 99% confidence level to conclude there is a reduction in the mean energy consumption.

Now, the question remains how large of a sample do you need? The answer depends on a number of factors. It's helpful to know what your budget is in planning this type of experiment because often time budgets dictate how much you can spend so sometimes you may be advised to sample as many households are you budget allows. Disregarding cost for the moment, you should try to select a sample size that provides you with sufficient power to detect true differences in mean energy consumption when they exist. Many statisticians select 80% power, which means there is an 80% chance of rejecting the null hypothesis when the alternative hypothesis is true (i.e. there is reduced energy consumption from the new bulb). Then you'll have to make some educated guesses on how much energy, on average the homes actually consume in time periods 1 and 2 (call these $\mu_1$ and $\mu_2$). You might be able to get estimates from your existing data and experiments with your bulb connected to a energy meter in the lab for example. You'll also need to determine an estimate of the standard deviation of the differences in energy consumption $\sigma_Y$. You'll then use these parameters for determining the sample size, $n$. The easiest way to compute this is to use any of the available online sample size calculators. I'll use this one here: https://www.stat.ubc.ca/~rollin/stats/ssize/n2.html. There is a reference on the web page to the formula used to compute this if you are interested in that.

So for example, let's assume $\mu_1=52$ and after introducing the bulb you expect the mean energy consumption to fall to $\mu_2=50$ and there is a standard deviation of $4$. Let's also say you'll want to test your hypothesis at the 99% confidence level. Then you simply input these parameters into screen as shown below and click calculate:

After doing that, you'll see that the sample size required is 63. You'll then need to sample 63 households. Keep in mind that this assumes 100% participation and compliance with every home in your experiment. If you suspect 10% of the participants might drop out, you should inflate this sample size estimate by 10%.

That all being said, this test assumes that there are no confounding variables between the two time periods. This may not be the case. For example an extremely hot summer in time period 2 may mean that more households run their AC units more often in time period 2 than in time period 1 and this is actually what caused the increase in energy in time period 2, rather than the bulb. To get around this, I'd recommend using a regression model and randomly assign half the households to get the bulb in time 1 and the other half in time 2. Then you can take this into consideration in building a regression model to better control for these types of external factors, but this is beyond the scope of my answer.

You may also find this reference quite detailed and provide methods for computing sample sizes in R: https://stats.idre.ucla.edu/r/dae/power-analysis-for-paired-sample-t-test/

Answer 2

Since no one, besides StatsStudent, provided an answer, I guess I'll answer it myself. I think I understand it now.

If I'm interested in determining the minimum sample size needed for the mentioned example with three light bulbs $A, B$ and $C$ and with the $K$ behaviors obtained from clustering (let us assume for this example that $K=3$ and that we label it as $k_1,k_2$ and $k_3$), the Multi-way ANOVA should be considered, more specifically a two-way ANOVA. Why? Because we have two factors: one for the light bulb $\{A,B,C\}$ and another with behaviors $\{k_1,k_2,k_3\}$ and two-way ANOVA is an extension of the one-way ANOVA that examines the influence of two different categorical independent variables (these two factors previously mentioned) on one continuous dependent variable (energy consumption).

For this calculation, softwares like $\texttt{G*Power}$ or $\texttt{R}$ can be used. For the latter one, the function $\texttt{wp.kanova}$ from the $\texttt{WebPower}$ package can be used. To determinate the minimum sample size, one has to give for input: Numerator degrees of freedom, Effect size, Number of groups, Significance level, and Statistical power. All of these inputs are directly known except for the effect size. For the effect size, one needs a previous trial study to calculate it or we can simply assign a guess value for it (carefully). The $\texttt{R}$ function will return the total number of samples, that we will need to divide with the number of groups and, thus, we will have the number needed for each group (in this case there are $3 \times 3 =9$ groups).