I had just conducted a test where the customer wants to see a 20% improvement in perforation length from a baseline perforator. The baseline perforator has not been tested in the customer's specific conditions in which he will eventually perforate in.
The customer plans to shoot 40,000 perforations in the future. I'm not sure if that bit of information is important but I wanted to be sure that it is understood that the true population mean of perforation length does not exist yet.
Under the specific conditions tailored for the customer the baseline perforator was shot 3 times with the following results: [7.68, 8.72, 8.08] inches.
Three new perforator designs were constructed in hopes to out perform the baseline perforator and each design was shot 3 times with the following results:
- D1 [8.04, 8.93, 10.05] inches
- D2 [11.15, 6.91, 9.21] inches
- D3 [7.1, 7.6, 7.02] inches
As I am relatively new to statistics I was wondering if some of you professionals can help me out with what information can be gained from these tests results. If you can point me in the right direction as in what methods I should refer to in gaining these insights you suggest would be of much help.
I know that effort is encouraged on my part so I would like to include some of my initial attempts at inferential statistics and maybe you can correct me where I am wrong or include what more I should/can do.
Estimating Sample Size Requirements: Solving for the sample size requires the population standard deviation, $\sigma$. Most often we do not know it so we have to use an estimate or “planning value” in its place. Here are a few options:
- Estimate σ from previous studies using the same population of interest
- Conduct a pilot study to select a preliminary sample. Use the sample standard deviation from the pilot study
- Use a judgment or “best guess” for $\sigma$. A common “guess” is the data range (high value – low value) divided by 4
The question I asked is: “What minimum sample size is necessary to produce 95% confidence (assuming a normal distribution) that the sample mean is $\pm$ 1 inch of the true population mean, $\mu$?”
Since we do not have an estimate of $\sigma$ from previous studies we will go with option 2 and consider our information as pilot studies.
Test charge Design 2 perforation length data is as follows: [11.15, 9.21, 6.91] inches. The sample mean is 9.09 inches and sample standard deviation is 2.12 inches. How large a sample should be shot to provide a 95% confidence interval with a margin of error (E) of 1 inch, assuming the population standard deviation is equal to the sample standard deviation?
$$n = \frac{(z_{\alpha /2})^2 (\sigma)^2}{E^2}$$
$$n = \frac{(1.96)^2 (2.12)^2}{(1)^2}$$
$$n = 17$$
Interpretation: To have 95% of our sample means contain the true population mean, $\mu$, we need a sample size of 17 shots. Or, 95 out of 100 tests made up of 17 perforation shots will contain within its average length $\pm$ 1 inch the perforation design’s true average perforation length.
It is not clear to me, however, how to move forward with this result. Does it mean I need to make an additional 17 shots as it's own sample or do I need to make 14 more shots to add to the original 3 shots? How do I handle the new information? I would suspect that the standard error of the mean would narrow with more measurements and would thus adjust the required sample size?
If we do not assume that the population standard deviation is equal to the sample standard deviation we can set the planning value for the population standard deviation as the Range of the perforation lengths we observed divided by 4 (option 3). From there we can estimate the minimum number of shots that should be taken with the margin of error of 1 inch to contain the true population mean 95% of the time.
Test charge Design 2 perforation length data is as follows: [11.15, 9.21, 6.91] inches.
$$\text{Planning Value for Std. Dev.} = \frac{\text{Range}}{4}$$
$$\text{PV} = \frac{(11.15-6.91)}{4} = 1.06$$
$$n = \frac{(z_{\alpha /2})^2 (\sigma)^2}{E^2}$$
$$n = \frac{(1.96)^2 (1.06)^2}{(1)^2}$$
$$n = 4.3$$
Interpretation: To have 95% of our sample means contain the true population mean, $\mu$, we need a sample size of 4 shots. Or, 95 out of 100 tests made up of 4 perforation shots will contain within its average length $\pm$ 1 inch the perforation design’s true average perforation length.
So which would be the more appropriate method (option 2 or 3)? And how should that information be used? In both cases more shots were required. But what does it tell me if the opposite occurs? e.g. for charge Design 3, using both option 2 and 3 method, I got n = 0.37 shots and n = 0.08 shots respectively.
Lastly, I tried to estimate what the %chance that the new charge design's true-population-mean-perforation-length will meet the required improvement. Taking the mean of the baseline shots to be 8.16 inches and adding 20% to it I found the goal length to be $\approx 9.792$ inches. By taking the sample means of each design we see that none of them averaged to meet the required length. However, I found the 95% confidence interval using a t-distribution for Design 2 to be [3.81 to 14.36] inches. Using excel Goal Seek I believe there is a 31% chance that Design 2's true perforation mean length will meet the 9.792 inch requirement. I'm not sure if my method was correct but maybe you can tell me if I am wrong or right or if interested, I will spell out how I went about that estimation.
