What methods can I research to help me estimate a probability

Question

This is my first question on here, I hope it goes well... I need to estimate the probability of an event that I don't have historical data on, but I do have data on similar/comparable events. I hope this is not too general. For example, take the damage rate of one item with one particular; this is a simple count, did the item get damaged--Yes/No and convert to a rate. I believe this data is characterized by a binomial distribution. Is there a method that I can apply to estimate the damage rate of another similar item based off the item I do have data on??

Answer 1

A sample represents the population it is drawn from. The question becomes how much similarity there is between the population you observed, and the population you didn't observe but want to reason about. Suppose you have a box of Item X and can observe the damage rate. You may or may not be able to use that as an estimate for Item Y, depending on how similar they are. If Item X and Item Y are similar transistors both made by a particular company in a particular time period with the same equipment, it's reasonable to guess that they might be damaged at the same rate - you're effectively suggesting that Item X can be used to reason about the wider population of transistors, rather than just the population of Item X transistors. But if Item X is a transistor, and Item Y is a chicken egg, you'd expect entirely different processes to be responsible for damage, so you couldn't reasonably guess at the damage rate of one from the damage rate of the other.

There's nothing in the data itself that can tell you whether the observed rate is a decent proxy for the unobserved rate. It will require understanding of the underlying processes, and how similar those are between the two items you want to compare. The damage rate of transistors made by one company will probably be a decent guess at the damage rate of other transistors from that company, a less useful guess at the damage rate of transistors made by a different company, and a useless guess at the damage rate of chicken eggs.

Answer 2

Suppose items come in boxes of $n = 24,$ then the number $X$ of defective items in a box might be modeled as $\mathsf{Binom}(n=24, p = .01)$ so that you'd expect an average defective rate of $\mu = np = 0.24$ per box.

Then if you got $y = 6$ defects in ten boxes, you might wonder if you got a shipment of boxes with higher than the usual defective rate. It would be fair to text the the null hypothesis $H_0: p = 0.01$ against $H_a: p > 0.01$ based on $y = 6$ defects in $n = 240.$

In R, an exact binomial test would look like this:

binom.test(6, 240, p=0.01, alt="g")

        Exact binomial test

data:  6 and 240
number of successes = 6, number of trials = 240, 
 p-value = 0.03489
alternative hypothesis: 
 true probability of success is greater than 0.01
95 percent confidence interval:
 0.01094225 1.00000000
sample estimates:
 probability of success 
                  0.025 

The P-value $0.03489 < 0.05 = 5\%$ indicates significance at the 5% level. Also, the lower bound on the one-sided confidence interval for $p$ is $0.011 > 0.01,$ also suggesting incompatibility of six defects in 240 items as inconsistent with a 1% defect rate.

Notes: (1) A direct computation of the (one-sided) P-value in R is shown below. $P(X \ge 6; n=240, p=0.01)$ $= 1 - P(X \le 5; n=240, p=0.01)$ $= 0.03488787.$

1 - pbinom(5, 240, .01)
[1] 0.03488787

(2) If you're dealing with thousands of items, you might want to use a Poisson model.

For values of interest here, the distributions $\mathsf{Binom}(240, 0.01)$ and $\mathsf{Pois}(2.4)$ are very nearly the same.

R code for figure:

x = 0:10
pdf.b=dbinom(x, 240, .01)
pdf.p=dpois(x, 2.4)
hdr = "Distributions of BINOM(240, 0.01) [bars] and POIS(2.4) [dots]"
plot(x, pdf.b, type="h", lwd=3, col-"blue", main=hdr)
 points(x, pdf.p, pch=19, col="orange")
  abline(h = 0, col="green2")
  abline(v = 0, col="green2")

Answer 3

If you have more than 1 other experiment (e.g. you have damage rates for more than 1 other product), you might want to fit a partially pooled hierarchical logistic regression, and estimate the variance of the defect rate across products. This can give insight into where you might expect to see the damage rate of a new, unknown product lie.

Here's a link to a discussion about the use of hierarchical modeling for extrapolation.