Back Pedaling Honestly, I'm not even sure if this is the right question but I haven't been able to come up with anything that makes more sense so I'd appreciate some help. This is a real problem, not an exercise. Actually, I hope I'm missing something obvious and will feel dumb once someone puts me on the right track.

What's the question? What I'm asking is: Am I right? Is this the correct way to view the problem and does my approach make sense? If you think it is (and does), let me know. Even better if you can tell me why. Put another way, I am asking for a review. If you want to suggest a better title, please do.

This is a real problem so there are several extensions. If my initial approach is right, it leads to more questions. (doesn't everything?)

BruceET gives a nice solution but see my comment: Doesn't that result in two p-values to consider?

So here goes:

You are about to do a job (like some car repair) for which you have good reason to believe you will not need more that $n$ things, all the same. You have done this job before, and you do many kinds of jobs, some of which require the same things. If you don't have $n$ already, you will buy more. But more often than not, you don't use all $n$ things. You have records of the number of things you used each time you did the job.

More concretely: You are repairing a machine and you know how many parts it has. You know something is broken, and you have to shut down the machine to repair it. Until you start, you don't know how many parts you will need to replace, but you know the maximum. ($e.g.$ while replacing a manifold gasket, you may break some bolts...or not).

Once you start, you want to be very certain that you have everything you need, (or you may be walking to the auto parts store) but you would like to avoid buying more than you need. So you want to adjust $n$ based on experience so that $n=k$ where $k$ is the number of things you end up using.

This seems like repetitions of a binomial experiment to me (but I could be completely off), where $n$ is the number of attempts (to use the thing) and $k$ is the number of things you ended up using (success). All I can find is how to solve for $p$ given $n$ and $k$. It also seems like a very Bayesian sort of problem, but I think that would need many repetitions of the job to converge on a satisfactory solution.

My proposed solution

I'm thinking of finding the binomial confidence interval, given $n$ and $k$ and if that interval is not too close to 0 or 1 (either side), reducing $n$ until it is. Too close means either there isn't enough data for a "good" test or $n$ is fine as it is. Either way: don't change $n$. No, I'm not precisely sure what "too close" means.

But this seems kind of sketchy. So I'm asking for comments/suggestions.

By the way, if I'm thinking about it right, for any particular kind of job, it doesn't matter how many times the job was done, only the total of n and k across all the jobs (like flipping a coin multiple times, taking a break and then flipping the same coin more times).

And furthermore...

The problem almost seems like a Poisson process but it isn't occurring over time or space (or any interval), each job is an event. Note that this seems related to failure rates but isn't quite. We already know something has failed, we just don't know how many things we'll want to replace.

Extension 1: You do many different jobs, each of which require multiple things.

Extension 2: Many of the "things" are indeed discrete (like bolts) but some are not (like gallons or feet). For those, I'm wondering if it really matters. Aren't 100 bottles of beer 100 discrete things?

Extension 3: Ultimately this is an optimization problem because if you don't have what you need, you lose money until you can finish the job.

Note: I hope I'm not violating some SE rule here with the extensions, if I get a good answer to the main problem, obviously I'll accept it and then maybe ask the extensions as separate questions. I just thought it would be good to give the whole picture.

  • 3
    $\begingroup$ This is hard to follow. If this is not a textbook exercise, but is a real problem, please add the concrete details from your actual problem. That will facilitate understanding & lead to better answers. $\endgroup$ – gung - Reinstate Monica Dec 29 '18 at 18:52
  • 3
    $\begingroup$ 1. Given that you can't expect to exactly hit the right number, you could consider formulating a loss function based on the relative costs of having more than you need against the cost of having to go back and buy more (these two costs - of being above and below - will be different in form). 2. I think a binomial is unlikely to be a suitable model, since (a) different parts will have different probability of needing to be replaced, and (b) there's likely to be a degree of dependence between some parts. $\endgroup$ – Glen_b Dec 30 '18 at 0:14
  • $\begingroup$ yes, that's the extensions (1 & 3). Please elaborate? (see my comment to @BruceET For ext 2, I'm calling it the "100 bottles of beer on the wall" rule. As long as you always finish a bottle you took down, I think beer is discrete (or fuel, or feet of pipe...). Maybe has to be same brand of beer to be exchangeable, maybe not. $\endgroup$ – M T Dec 31 '18 at 22:12
  • $\begingroup$ Please reduce this problem to something with less text. $\endgroup$ – Sextus Empiricus Jan 4 '19 at 10:26

I'll make some assumptions for one fixed kind of job. If they're not realistic, that may help you ask a more targeted question.

You repeatedly encounter a binomial random variable $X \sim \mathsf{Binom}(n = 10, p),$ where $n$ is the maximum number of items you might need for the job, $X$ is the number actually required, and $p$ is unknown. [This is realistic if items fail independently of one another.]

You have data from $m = 100$ previous jobs of this kind. Out of the maximum number $mn = 100(10) = 1000$ of items you might have needed, you actually needed $700,$ so a reasonable estimate of $p$ is $\hat p = 700/1000 = 0.7.$

Based on the above, a traditional ('Wald') 95% confidence interval (CI) for $p$ is of the form $\hat p \pm 1.96\sqrt{\hat p(1 - \hat p)/1000},$ approximately $(.67,.73).$ [Possible variations: With substantially less preliminary data, an Agresti-Coull style of CI might be better, but with amount of data assumed above, the difference between the two styles of CI is negligible. Also with th amount of previous data assumed above, a Bayesian interval estimate, based on an uninformative prior distribution such as $p \sim \mathsf{Beta}(.5, .5)$ or $p \sim \mathsf{Unif}(0,1),$ would be numerically about the same.]

qbeta(c(.025, .975), 700.5, 300.5)  # for Bayesian prior BETA(.5,.5)
[1] 0.6710503 0.7277938

Then, playing it safe by using the top end of the interval estimate, it would be sensible to ask whether $X = 8$ items would be enough for the next repair job of this kind: $P(X \le 8\,|\,p =.73) \approx 0.80.$ If that seems too risky, you might ask the probability that nine items would be sufficient; that probability is about $0.96.$ [Computations in R.]

pbinom(8, 10, .73)
[1] 0.7980705
pbinom(9, 10, .73)
[1] 0.9570237

Of course, you could change $n = 10$ to something else and imagine a different amount of prior data, making changes that I hope are obvious. The main issue is whether this very elementary approach provides a reasonable answer to your Question. If not, please edit your question or leave comments so that the next try to give an answer can be more useful.

| cite | improve this answer | |
  • $\begingroup$ On reflection, I realized your last step is the key, and not what I was suggesting. But then don't I end up with two P values to worry about (1 on the C.I. and one on the last step)? So what is the probability over all? $\endgroup$ – M T Dec 30 '18 at 0:05
  • 1
    $\begingroup$ You're correct that there is some uncertainty (.05) in the bound .73 and then additional uncertainty (.04) that 9 would be enough even if .73 is a reasonable bound. Not knowing how big your $m$ might be, it is difficult to know whether this approach would be useful. @Glen_b gives a hint toward an approach you might find satisfactory. // If this is an applied problem, it would help if you say more about what you really want to accomplish. If this is a textbook problem (or your fuzzy paraphrase of one), then stating the exact problem, level of the course, topics studied recently might help. $\endgroup$ – BruceET Dec 31 '18 at 19:10
  • $\begingroup$ Thanks for the added response. I'll prob reply to @Glen directly but in a nutshell, he went right for my "extensions." This was for a feasibility study, due today, that I just declared feasible. Perhaps I should edit the question a bit, the rest will depend on if we can sell it. If we do, I'm sure I'll have lots more questions and expect that SE/CV will be right there with me. An interesting complexity for the optimization is that the other side of the equation is millions of $'s/day, so anything traditional will swamp out the buy side. Also, "work orders" will need work to line up... $\endgroup$ – M T Dec 31 '18 at 22:07
  • $\begingroup$ btw, since there is no closed form on the c.d.f., and I couldn't get Hoeffding to do anything useful, I just checked binom(0.9*n, n, p), say, for p>0.99 and the results over several thousand parts are quite reasonable. Time to search for an "exact" p later, I hope. Thanks for your help! $\endgroup$ – M T Dec 31 '18 at 22:24

If I understand your question properly, what you need is not a confidence interval for $p$ but a prediction-interval for $X$. Let $X_1, X_2, \dots, X_m$ be the number of parts you needed to use to repair a car in the last $m$ time. We know $X_i$ must take a value in $\{1,2, \dots, n\}$. For example, you may have data like $3, 1, 40, 23, \dots, 11$.

Since some parts in a car always broken together, Binomial assumption may not suitable in this case. However, we may assume each $X_i$ is independent to each other and is identically distributed. I guess your main interest is to know a prediction interval $[a,b] \subset [1, n]$ which have a coverage guarantee (let say 95%) such that $$ \mathbb{P}(X \in [a,b]) \geq 0.95 $$ where $X$ is a 'new' random number you will encounter in the next car repairment.

One possible way is to use the exchangeability assumption to build a prediction interval which is often called conformal prediction.

Let me explain one simplest way to build the prediction interval :

  1. Split data into two sets randomly. Let say $D_1 = \{X_{1,i}\}$ and $D_2 = \{X_{2,i}\}$ are the two sets.
  2. Calculate sample mean by only using $D_1$. Let $\bar{X}_1$ be the calculated sample mean.
  3. By using data in $D_2$, calculate upper and lower 0.025 quantiles of $\{X_{2,i} - \bar{X}_1\}$. Let $q_l$ and $q_u$ the calculated quantiles.

Then, it can be shown that the interval $[\bar{X}_1 + q_l, \bar{X}_1 + q_2]$ is a prediction interval with 95% coverage.

This is not an optimal one but is vaild. For technical detail and optimal interval, please see https://arxiv.org/pdf/1111.1418.pdf.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.