Let's say I have three tweets and those three tweets are all from either Mary or John. There is no possibility for mixed result.
index tweet prob_Mary prob_John 0 ... 0.5 0.5 1 ... 0.6 0.4 2 ... 0.53 0.47
The probability of each tweet being from Mary is: 0.5*0.6*0.53 = 0.159 and probability of each tweet being from John: 0.5*0.4*0.47 = 0.094
In the beginning we declared that all the tweets must be from either Mary or John. Therefore probability of Mary being the tweeter: 0.159/(0.159+0.094) = 0.63 and probability of John = 0.37.
There is no problem with this approach as long as the number of tweets is relatively low. However if we increase the number of tweets, both probabilities (Mary & John) will converge to either (1.0 and 0.0) or (0.0 and 1.0). This is problematic because the original single-tweet-probabilities are just predictions/estimates made by a model. Therefore there will be bunch of huge prediction errors.
I could just take an average from the tweets (resulting 0.54 for Mary and 0.46 for John), but that doesn't really seem like proper way to deal with the issue.
How should I deal with this issue?