2
$\begingroup$

Suppose I have access to data on the energy consumption of a city with a population of 30000 persons. Imagine I want to test a new technology of smart energy, i.e if the energy consumption decreases with this technology. I need to subset the 30k to a smaller group G where they use the technology and compare it to the rest of the population (where they don't have this technology). Then I can make a hypothesis test where the mean of energy consumption is equal in both groups and the alternative hypothesis is the opposite. However, to make sure the technology was the reason the energy consumption was reduced and not only a coincidence, I need to get a significant level (e.g 0.05) of a certain level and a minimum size for the group G, right?

I also read that a power test needs to be done to avoid type I and type II errors. But is that really necessary? I'm new to this topic and this is very confusing to me because there are so many different formulas and I don't know which one is correct to determine the size of group G.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ You also need a variance estimate for houses with the new technology. And to say what power you want for your 5% level test. // It may be best to do a one-sample test comparing the mean energy usage of $n$ houses with new technology with the (known population) mean of houses without. $\endgroup$
    – BruceET
    Commented Nov 23, 2020 at 3:31
  • $\begingroup$ @BruceET how can I determine n? Do you have any references/bibliography about this? $\endgroup$
    – Numbermind
    Commented Nov 23, 2020 at 16:47
  • $\begingroup$ Give me variance estimate, desired power, and energy savings you want to detect, and can give a targeted answer. $\endgroup$
    – BruceET
    Commented Nov 23, 2020 at 18:35
  • $\begingroup$ Note that no amount of sample size can tell you that "the technology was the reason the energy consumption was reduced." A sufficient sample size can reveal significant association, supporting the notion that energy consumption is, in fact, lower in homes with the technology. But that alone does not support a causative interpretation that the technology is responsible for the difference in energy consumption. You can imagine a scenario where the technology is for some reason installed only in small homes with low consumption, but the technology itself doesn't affect consumption at all. $\endgroup$
    – Nuclear Hoagie
    Commented Nov 23, 2020 at 19:44

1 Answer 1

Reset to default
2
$\begingroup$

This isn't a direct answer to your question, but it does illustrate what information you need to input to a 'power and sample size' procedure to get the required sample size.

Let's suppose the current number of "energy units" per day for 30,000 households is $100.$ With new technology you expect energy consumption per household to be normally distributed with mean $\mu < 100$ with $\sigma = 20.$ You hope to have power 90% of detecting decrease of as much as $5$ energy units. So if the particular alternative $H_a: \mu = 95$ is true you want the rejection probability to be $0.9 = 90\%.$

To be sure, some of this "information" may be unknown and speculative, but all of the above is necessary input. (You can experiment with slight variations of the input to see the effect of the output.)

Here is output from a recent release of Minitab to illustrate:

Power and Sample Size 

1-Sample t Test

Testing mean = null (versus < null)
Calculating power for mean = null + difference
α = 0.05  Assumed standard deviation = 20

            Sample  Target
Difference    Size   Power  Actual Power
        -5     139     0.9      0.901145

So in this hypothetical scenario you would need a sample size of $n = 139$ to get the desired power. The following graph shows power for detecting a decrease of $5$---along with other possible decreases.

enter image description here

Under my assumptions, it seems feasible to install the new technology in about 140 houses and to do a one-sample t test of the results $H_0: \mu=100$ vs. $H_a: \mu < 100$ at the 5% level.

Notes: (1) For normal data, such computations use a non-central t distribution with degrees of freedom $n - 1$ and a non-centrality parameter that depends of desired power, size of difference to detect, and anticipated population SD for the $n$ observations.

The crucial fact is that $n = 129$ observations suffice to give 90% power of a difference that is $5/20 = 1/4$ as large as the anticipated SD.

You can search this site, and the Internet for technical explanations at your level. This recent Q&A may be helpful.

(2) Many statistical computer programs have 'power and sample size' procedures. There is a library in R with such procedures for a variety of types of tests. There are online sites for power and sample size computations, but not all of those are reliable.

(3) In R, the probability functions dt, pt, and so on have a (seldom used) parameter 'ncp` for the non-centrality parameter.

Simulation in R: With 100,000 iterations, one can expect about two-place accuracy. So the simulation is in essential agreement with the Minitab output.

set.seed(1121)
pv = replicate(10^5, t.test(rnorm(139, 95, 20), mu=100, alt="less")$p.val)
mean(pv <= 0.05)
[1] 0.89914
Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you very much! Can you recommend to me a book/reference that has this procedure? Also, how can I estimate the standard deviation (like you said e.g 20). $\endgroup$
    – Numbermind
    Commented Nov 24, 2020 at 9:09
  • $\begingroup$ A reference with the actual formula would also be great! Thanks in advance $\endgroup$
    – Numbermind
    Commented Nov 24, 2020 at 10:03
  • $\begingroup$ After I got the sample size (e.g 139), say I install the technology for them. After a period I see the energy decreased to the estimated 95. I can say with 95% confidence that the technology made the energy consumption decrease? $\endgroup$
    – Numbermind
    Commented Nov 24, 2020 at 10:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.