I am having a hard time to exactly name what I am looking for (I am quite sure it already exists out there...) so I'll start with a concrete example:
I have a population of discrete colours (red, green, blue, ...) of unknown distribution from which I draw a rather small sample [red, red, green, yellow] that is the basis for my simple empirical probability model.
In the end I am interested in the empirical probability p(x = red), but since my sample size is so small, I also want to know the distribution of p(x = red).
I tried bootstrapping on p(x) which is probably exactly what I want, but it is too computational expensive for my application. Given the sample size, and p(x = red) I guess there is a closed form solution to my problem...
Or maybe in other words: What is the distribution of p(x) in relation to the sample size and p(x) itself?
Background: I am building Markov Chains on sampled data. Since I can't make any assumptions about the underlying distribution, I use the empirical probability model based on the sample data. The model is built and evaluated in real time, that's why bootstrapping is no option.
update For a sample of [6x red, 6x green, 6x blue, 6x yellow] the bootstrapping of p(x = red) looks like this.