How to find the threshold between two histograms at fixed false positive rate? I have two histograms with equal bin size that lie in the same general range. You can imagine them looking like this example I've borrowed from https://mpatacchiola.github.io/blog/2016/11/12/the-simplest-classifier-histogram-intersection.html:  
Let's say, the blue-ish histogram corresponds to true examples, while the red-ish histogram corresponds to false examples. I want to find a threshold that can be used to predict true examples or false examples in the future. If you want, this is a machine learning problem where I want to find a binary classifier for one-dimensional data. 
Obviously, a naive and straight-forward choice would be to find the local minimum in the middle of the two histograms and cut right through it. How can this be solved efficiently? My first idea would be to fit two Gaussians and intersect them. 
The whole thing gets more interesting if you additionally seek to fix the false positive rate (corresponding to the red-ish bins located to the right of the threshold). How can the FPR be put into the equation?
 A: Not exactly what the original question asked for...
import math
import numpy as np
import scipy.special
import scipy.stats
import matplotlib.pyplot as plt

mu_1 = -4
mu_2 = 4
sigma = 2

t = np.linspace(-10,10,1000)

# https://mathworld.wolfram.com/NormalDistribution.html
# See formulae 8,9,10 in
# Weisstein, Eric W. "Normal Distribution." 
# From MathWorld--A Wolfram Web Resource. 
# https://mathworld.wolfram.com/NormalDistribution.html
def area_under_gaussian_at_left(t,mu,sigma):
    a = t-mu
    b = math.sqrt(2)*sigma
    return .5*(1+scipy.special.erf(a/b))

def area_under_gaussian_at_right(t,mu,sigma):
    return 1-area_under_gaussian_at_left(t,mu,sigma)

false_positive = area_under_gaussian_at_right(t,mu_1,sigma)
false_negative = area_under_gaussian_at_left(t,mu_2,sigma)

threshold = t[np.argmin(false_negative+false_positive)]

print('Threshold minimizing the sum of false positives and false negatives: {0}'.format(threshold))

fig, ax = plt.subplots(1)
ax.plot(t,scipy.stats.norm.pdf(t,mu_2,sigma),
t,scipy.stats.norm.pdf(t,mu_1,sigma)
)
ax.axvline(threshold,color='black')
ax.set_title('The black vertical line is the threshold that minimizes\nthe sum of false positives and negatives: {0}'.format(threshold))
ax.set_xlabel('feature')
ax.set_ylabel('Probability density')
ax.legend(['class 1','class 2'],loc='upper right')
fig.savefig('far_vs_near.png',bbox_inches='tight')

fig, ax = plt.subplots(1)
ax.plot(
t,false_positive,
t,false_negative,
t,false_negative+false_positive
)
ax.set_xlabel('Threshold value for feature')
ax.set_ylabel('Proportion of falses (1 means 100%)')
ax.legend(['False positives','False negatives','False negatives plus false positives'],loc='upper right')
fig.savefig('falses.png',bbox_inches='tight')



A: It's actually more of a problem in Statistics. 
A little bit of theory:
Consider the following formulation using hypothesis testing, which you're hopefully familiar with.
In the general setting, we observe examples $x_{1},...,x_{m}\sim f\left(x\right)$ and we have two "candidates" for the density function  $f$ which the examples came from: $f_1$ and $f_2$.  In the language of hypothesis testing, we have $H_{1}$ and $H_{2}$ and we want to distinguish between them.
For a simple case, which suits your scenario better, consider the case of a single example (the case for a general $n$ is very similar but it's a little bit different than what you described since it assumes all examples come from either $f_1$ or $f_2$).
Given an example $x$, we would like to choose the distribution that is most probable: 
$prediction = argmax_{i=1,2}\mathbb{P}\left(H_{i}\mid x\right)$
Equivalently, we can choose $H_1$ $\iff$ $ \frac{\mathbb{P}\left(H_{1}\mid x\right)}{\mathbb{P}\left(H_{2}\mid x\right)} > 1$. 
Using Bayes theorem to simplify the above expression, 
$ \frac{\mathbb{P}\left(H_{1}\mid x\right)}{\mathbb{P}\left(H_{2}\mid x\right)} = \frac{f_{1}\left(x\right)}{f_{2}\left(x\right)} \cdot \frac{\mathbb{P}\left(H_{1}\right)}{\mathbb{P}\left(H_{2}\right)}$
Taking logarithm of both sides gives a nice decomposition (this step is not mandatory, but usually done for further analysis of the general case):
$\log\frac{\mathbb{P}\left(H_{1}\mid x\right)}{\mathbb{P}\left(H_{2}\mid x\right)}=\log \frac{f_{1}\left(x\right)}{f_{2}\left(x\right)} + \log \frac{\mathbb{P}\left(H_{1}\right)}{\mathbb{P}\left(H_{2}\right)}$
$\log \frac{\mathbb{P}\left(H_{1}\right)}{\mathbb{P}\left(H_{2}\right)}$ is known as the log likelihood ratio; it's fixed and signifies the initial inclination (without seeing any data) to choosing $H_1$; $\log \frac{f_{1}\left(x\right)}{f_{2}\left(x\right)}$ can therefore be seen as the additional information that example $x$ gave you regarding choosing $H_1$. 
(In the general case, we can go on to show that - as one would anticipate - when you see more examples, the probability of choosing the correct density goes to 1.)
Practical Usage: 


*

*Use the information you have (histogram, etc) to fit a density estimator for your two candidate functions. This can be done in two lines of code in Python, R, etc.

*Predict: $\hat y = argmax_{i=1,2}\hat f_{i}(x)$
A: You could analyze this graphically with a ROC curve like the image below.

The image shows that there is not a single best point for the threshold. You can have go for a better true positive rate, but this comes at the cost of an increase in the false positive rate.
The optimal threshold will eventually depend on some quantitative expression comparing the pros and cons as a function of the threshold. Reducing the threshold will improve the sensitivity which is beneficial but at some point, the costs of the decreased accuracy will weigh stronger. Where this point is, depends on the particular function weighing the costs and benefits of the particular case (there is not a general answer).
If you have an expression for the costs/benefits of the wrong/correct classification (and also an estimate of the distribution of true/false cases among the population), then you can solve it computationally and potentially you can use some estimate of a parametric distribution but you could also use the empirical distribution.
A: We can re-frame this problem in a different way. It is a very common problem to find a threshold to distinguish two classes in a bimodal distribution like this. Even in image processing and computer vision it is a common problem to find a threshold in a bimodal distribution to make a grayscale image to a binary image.
For these kinds of problem, one of the prominent ways to find a single threshold is to use Otsu Method. This method has a limitation:

Limitations:
Otsu's method performs well when the histogram has a bimodal distribution with a deep and sharp valley between the two peaks.

But as your histogram is clearly a bimodal distribution, so it is not an issue for you. You can find Otsu Method and various other thresholding techniques here: OpenCV | Image Thresholding.
