1
$\begingroup$

Basically, let's assume I have a simple rules-based function/model (if weight >= 150) -> return true. Simple binary answer (true or false) from a single feature input.

If I have a range of samples/population values that this rule was derived from (i.e. 50 - 400), are there techniques to derive a probability distribution or "prediction score". i.e. 150 would be 0.5 but 400 would really be 1.0. I am looking for any mathematical techniques to convert this into a "interpret-able probability score".

My second (related) question is best practices around combining (ensemble) of probability based models with rules-based models. Doing simple prescriptive engines works to a degree, but what I am looking for is for example "stacking" 2 probability models and 2 rules-based models based on a common set of outputs. Obviously, if a rules based model can have an "interpret-able" associated probability this makes it a lot easier.

$\endgroup$
  • 1
    $\begingroup$ What probability are you looking for? It sounds like you are wanting to determine the probability of the value being over the threshold based on its value. This is identical to the result of the logic operator. This makes me assume you want something different, could you spell out exactly what the probability is of? Or are you wanting to estimate the probability of exceeding the threshold based on other information? $\endgroup$ – ReneBt Nov 14 '18 at 6:03
0
$\begingroup$

If I understand correctly, the problem has an easy solution (the first one). I will use Python to show the solution.

The list of numbers is:

numbers = [i for i in range(50, 401)]

And you have a function which returns True or False depending on a threshold.

def function(value, threshold):
    if value >= threshold:
        return True
    else:
        return False

Although it is better if you use numbers $0$ and $1$.

def function2(value, threshold):
    if value >= threshold:
        return 1
    else:
        return 0

If you want the portion of the list that is greater than the threshold, you can use

pr = []

for i in numbers:
    if function2(i, 150) == 1:
        pr.append(1)

print(sum(pr) / len(numbers)) # 0.71

Unfortunately, that portion changes if the distribution of the numbers is different.

Figure 1 Figure 1

import random

more_nums = []

for i in range(200):
    more_nums.append(random.randint(240, 400))

pr = []

for i in numbers:
    if function2(i, 150) == 1:
        pr.append(1)

print(sum(pr) / len(numbers)) # 0.81

Or

Figure 2 Figure 2

more_nums = []

for i in range(200):
    more_nums.append(random.randint(240, 400))

pr = []

for i in numbers:
    if function2(i, 150) == 1:
        pr.append(1)

print(sum(pr) / len(numbers)) # 0.58

Or Figure 3 Figure 3

more_nums = []

for i in range(1000):
    more_nums.append(random.randint(200, 300))

pr = []

for i in numbers:
    if function2(i, 150) == 1:
        pr.append(1)

print(sum(pr) / len(numbers)) # 0.92

If you want the probability distribution, for example, of Figure 3, you would have to use other tools to evaluate if that distribution is close to what you are looking for.

 Figure 4

enter image description here Figure 5

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.