I did not really know how to choose an adequate title for this question, so please feel free to change it.

I have a weird case wherein frequentist and Bayesian philosophies come together. I am dealing with a data set of observations on persons who were faced with a situation in which they could either perform an action or could decide not to. The data set consists of several persons with a varying number of observations on that person.

In a frequentist approach, I could say that the probability of a certain person performing that action is the number of times the person performed the action given the situation divided by the number of times the situation occurred. So: if the situation in which the person could perform the action occured $23$ times, and this person has performed the action $8$ times, we could say the probability of this person performing this action in that situation is $\frac{8}{23} = .348$.

Now, when I encounter a new person (not in the data set I already have), I want to have a reasonable estimate of the probability of this person performing this action in the said situation, given very few or no observations at all.

What I do (and what works) is:

• I consider my data set of historical observations and I leave out all persons with less than $x$ observations (in my case: 100 observations).
• I fit a mixture of gamma distributions to my data set (since there are different clusters of persons in my data set) using EM.
• I use Bayes' theorem (with a conjugate prior, my fitted Beta distribution is the prior) and update using Bayesian inference to get my probability estimate (and corresponding credible interval) for the newly observed person (of this person performing this action in the said situation).

However, I do not like that I am cutting out all persons with a lower than $x$ number of observations in my prior data set. In my particular case, persons with lower amounts of observations tend to belong to a different cluster compared to persons with a very high amount of observations. So: there is a correlation between the amount of observations and the probability I am trying to calculate! I feel like my prior is biased if I cut out some people (even if they have very few observations and are thus very unreliable).

My question: is there any alternative to my method of cutting out persons with observations below a certain number $x$? Can I maybe somehow still take these persons into account and define their (frequentist) probabilities relative to each other? Can I weigh them into the prior Beta distribution using their credible intervals calculated relative to the other players in the data set? Maybe I should use some sort of regression and factor in another variable for the number of observations..? Or is any other method than the method I am using just impossible?

I did not really know how to choose an adequate title for this question, so please feel free to change it.

I have a weird case wherein frequentist and Bayesian philosophies come together. I am dealing with a data set of observations on persons who were faced with a situation in which they could either perform an action or could decide not to. The data set consists of several persons with a varying number of observations on that person.

In a frequentist approach, I could say that the probability of a certain person performing that action is the number of times the person performed the action given the situation divided by the number of times the situation occurred. So: if the situation in which the person could perform the action occured $23$ times, and this person has performed the action $8$ times, we could say the probability of this person performing this action in that situation is $\frac{8}{23} = .348$.

Now, when I encounter a new person (not in the data set I already have), I want to have a reasonable estimate of the probability of this person performing this action in the said situation, given very few or no observations at all.

What I do (and what works) is:

• I consider my data set of historical observations and I leave out all persons with less than $x$ observations (in my case: 100 observations).
• I fit a mixture of beta distributions to my data set (since there are different clusters of persons in my data set) using EM.
• I use Bayes' theorem (with a conjugate prior, my fitted Beta distribution is the prior) and update using Bayesian inference to get my probability estimate (and corresponding credible interval) for the newly observed person (of this person performing this action in the said situation).

However, I do not like that I am cutting out all persons with a lower than $x$ number of observations in my prior data set. In my particular case, persons with lower amounts of observations tend to belong to a different cluster compared to persons with a very high amount of observations. So: there is a correlation between the amount of observations and the probability I am trying to calculate! I feel like my prior is biased if I cut out some people (even if they have very few observations and are thus very unreliable).

My question: is there any alternative to my method of cutting out persons with observations below a certain number $x$? Can I maybe somehow still take these persons into account and define their (frequentist) probabilities relative to each other? Can I weigh them into the prior Beta distribution using their credible intervals calculated relative to the other players in the data set? Maybe I should use some sort of regression and factor in another variable for the number of observations..? Or is any other method than the method I am using just impossible?