Applied statistics to find the minimum load for power factor floor

Question

The problem I’m trying to solve is “How do I figure out how much gunpowder should I put into a cartridge so that I can give myself a good probability of making the minimum power factor?”

I compete in USPSA/IPSC which requires that a competitors rounds make a minimum power factor. Power Factor is computed to be the FLOOR(average bullet velocity * bullet weight) / 1000) where velocity is in feet per second, and bullet weight is in grains. Note the use of FLOOR. No rounding is done. Only the integral part of the computation is used. The higher the power factor, the higher the felt recoil and harder it is to quickly do follow up shots. Since the sport is about firing shots as quickly and as accurately as possible, the lower the recoil the better.

Different divisions within the sport have different power factor floors, but the particular division I compete in has a minimum of 165 Power Factor. The bullets I use are 180 gn bullets, and vary by about +/- 0.2 gn and is normally distributed.

What makes this an interesting problem (and move it out of my meager stats and probability skills) is the testing procedure during a major match. Random sample of 8 rounds are collected. Of the 8 rounds, one is taken apart and the bullet is weighed for use in the formula above. Next, 3 rounds are fired and the average velocity is used. If the resulting power factor is below the minimum, then another 3 rounds are fired. The average of the 3 fastest velocities from the 6 rounds fired is now used to compute the power factor. If the resulting power factor is still below the floor, then the shooter has the option of having the last round taken apart and weighed or the last round fired. If the bullet is taken apart and it is heavier than the first bullet, the heavier weight is used to compute the power factor. If the last round is fired, the average of the 3 fastest velocities from the 7 rounds fired is used to compute the power factor.

To add spice to this problem, not all chronographs used to measure bullet velocities are created equal. The chronograph industry acknowledges that there can be as much as +/- 4% variance between chronographs of different brands. Even more interesting is that the rules allow for the same chronograph used for a particular match to have +/- 4% variance over the duration of a match. I don’t know if either of these 4% variances are normally distributed or not.

With my own chronograph, I test batches of a particular gunpowder load to get the average velocity and standard deviation. After statistical analyses of many different batches, I’ve confirmed that this data is normally distributed.

The way I’m currently determining my minimum load is by finding the load the gives me 165 < FLOOR( target * 179.9 / 1000) where target = (average velocity - standard deviation) * fudge factor. For fudge factor, I’ve unscientifically chosen 1.04. The 0.04 is the 4% variance between chronographs, but ignores the day-to-day allowable variance. I chose to just subtract just 1 standard deviation because it’ll only be 16% of the time that one bullet will be below the floor. In my mind, the probability of all the first 3 bullets all going below the floor is 0.16^3 which less than half a percent.

My specific questions are: Am I going about computing the target the right way? Should my fudge factor include another 4% for the day-to-day variance allowed? Is the 1 standard deviation too much or too little? How should I write the formula for my target?

Edit: After Srikant's initial response below let me add a couple of focusing questions and notes.

I understand figuring out the error due to my measurements. Not much problem there unless I get really sloppy with quality control or maintenance.

My grasp of probabilities is weak so please bear with me as I ask about computing probabilities:

1) One of the key issues I need to deal with is figuring out how to correctly compute the probability for the testing process around the 7 rounds. It's pretty straight forward to me for the case of the first 3 rounds: (probability that the round is below the power flaw)^3. How do I account for the next 3 rounds and the last round?

I can see computing the probabilities using combinations of bullets above or below the floor, but it's not quite a binary above or below. Let's assume that 6 rounds have been fired, and the average of the highest 3 rounds is 164.9. If the last round has at least 165.2, then the average of the highest 3 rounds out of 7 will be 165.

2) The other issue I need to deal with is figuring out how to account for the 4% variance between my chrono and the match chrono, and how the match chrono is allowed to drift by 4% from day to day. Do I just assume the worst case and make sure that I'm at least 8% above 165 -- that is my rounds are shooting at least 179 power factor? Or do I try to assume some kind of normal distribution over the two 4% variances?

Answer 1

I doubt that you can get an analytical solution to this problem. About the only step that is doable is the probability that the average of the first three velocities is under some cutoff and it is not (probability the one velocity is under the cutoff)^3 as you stated. If $V_i \sim N (m, s^2)$, then $\bar{V}_3 \sim N (m, s^2/3)$, so the probability that $\bar{V}_3$ is less than $k$ standard deviations above the mean is $P(\bar{V}_3 < m + k s ) = \Phi(k/\sqrt{3})$ where $\Phi$ is the normal distribution function. For example, for $k=0$: $P(\bar{V}_3 < m) = 0.5$, for $k=1$: $P(\bar{V}_3 < m + s) = 0.718$, etc.

The calculation involving the next three rounds is complicated by two things: it should somehow be conditional on the fact that the average was too low already, and then should only use the three highest values. While it might work out to be some triple integral, I don't think you want to use such a formula. After that the fact that you have a choice whether the bullet should be weighed or fired also probably depends on the previous values, and things get even more complicated depending on your strategy.

Even the simple multiplication of a random bullet weight and one random velocity is not as innocuous as it seems: the product will not be normal (or any other "regular" distribution). Its standard deviation can be approximated via standard error propagation, but calculating probabilities of falling below/above some cutoff is not straightforward.

In summary, I think you should write a little simulation program - you will get better answers faster. Note that for the chronometer you will have to define more exactly what does +/-4% mean.

Answer 2

You have a complex statistical problem and a complete analysis would be too long. However, I will suggest one idea that may perhaps help you to some extent.

You have already performed some calibration tests to assess the mean and standard deviation of the velocity of a bullet. However, I suspect that either your testing or your calculations or both are sub-optimal for the reasons I mention below.

The crucial point is this: The measured velocity of a bullet is a random variable that depends on three factors: (a) amount of gunpowder in the bullet, (b) random factors due to imperfections of the gun, and (c) errors because of chronograph inaccuracy. In other words, you can model the measured velocity of a bullet as follows:

$v_m = v(gp) + \epsilon_g + \epsilon_c$

where,

$v_m$ is the measured velocity,

$v(gp)$ is the true velocity of the bullet given that the bullet has $gp$ amount of gunpowder,

$\epsilon_g \sim N(0,\sigma_g^2)$ is the random error that arises due to imperfections of the gun and

$\epsilon_c \sim N(0,\sigma_c^2)$ is the error induced by the chronograph.

I think assuming that the errors are normally distributed with a mean zero and a finite variance is reasonable. Thus, we have:

$E(v_m) = v(gp)$

and

$Var(v_m) = \sigma_g^2 + \sigma_c^2$

The goal of testing is to get a sense of the average velocity and the standard deviation of the velocity of a bullet. You could simply compute the mean of the observed velocities to get a sense of the mean as $E(v_m) = v(g)$ but you cannot use the standard deviation of the observed velocities for setting your target as the the errors induced by the chronograph and the errors due to the imperfections of the gun are confounded.

Thus, you need to design your test to disentangle the variation induced by the chronograph and the intrinsic variation because of other factors. One way to achieve the above disentanglement is to perform a test as follows:

1. Start test 1 and choose a certain amount of gunpowder that you will fill up each bullet for this test.
2. Load the bullet with the amount of gunpowder you chose for this test in step 1.
3. Measure the velocity of the bullet.

4. Repeat steps 2 and 3 for a certain number of bullets.

5. Perform several tests along the above lines ensuring that you choose a different amount of gunpowder for each test.

Let:

$i$ index the bullets,

$j$ index the test.

Then you have a set of velocities indexed $v_{ij}$. Then, you can calculate the variance of the velocity (and hence the standard deviation) as follows:

1. Calculate the variance of all velocity measurements. Denote this number by $S_v^2$.
2. Calculate the variance of velocities for each test separately. Denote each test specific variance by $S_j^2$. Note that each of these variance measures is a measure of how much variation is induced because of the chronograph as the amount of gunpowder is the same for each test.

Then a measure of the variance of the velocity that you can attribute to the gun itself would be:

$S_g^2 = S_v^2 - \frac {\sum_j S_j^2}{J}$

Once you know $S_g$ you can use your existing formula and there is no need to worry about the additional variation induced in your measurements because of your own chronograph as that has been 'taken care of' in our analysis.

Hope that helps to some extent.