# Is it possible to define the mean of a varying distribution?

Suppose $(p_1,\ldots,p_k)$ be the vector of multinomial parameters and $$(p_1,\ldots,p_k)\sim \mbox{Dirichlet}(\alpha_1,\ldots,\alpha_k).$$ Let's define a function $f(p_1,\ldots,p_k) \in \mathbb{R}$. After collecting some observations $(x_1,\ldots,x_k)$ incrementally we have $$(p'_1,\ldots,p'_k)\Big|(x_1,\ldots,x_k)\sim \mbox{Dirichlet}(\alpha_1+x_1,\ldots,\alpha_k+x_k).$$ Imagine we calculate $f$ after each increment to the $x_i$'s. Can we calculate the mean over the outputs of $f$? Or, can we say anything about the changes between $(p_1,\ldots,p_k)$ and $(p'_1,\ldots,p'_k)$?

The mean of $f(p_1,\ldots,p_k)$ is defined in terms of the distribution of $(p_1,\ldots,p_k)$, namely $$(p_1,\ldots,p_k)\sim \mbox{Dirichlet}(\alpha_1+n_1,\ldots,\alpha_k+n_k).$$Hence, if the data $(n_1,\ldots,n_k)$ changes, so does the distribution of $(p_1,\ldots,p_k)$, and the means $$\int f(p_1,\ldots,p_k) \pi((p_1,\ldots,p_k)|(n_1,\ldots,n_k))\,\text{d}\mathbf{p}$$ differ. Averaging successive means as $(n_1,\ldots,n_k)$ grows does not make sense from either a probabilistic (this is no longer a mean against a fixed distribution) or from a Bayesian (the final posterior distribution contains all the relevant information) viewpoints.
• Thanks for your answer! Can we say anything about the changes between $(p_1,\ldots,p_k)$ and $(p'_1,\ldots,p'_k)$? In other words, can we say if we increase $\alpha_i$ by $n_i$ in the sampled multinomial vector the values would change by $\delta$ before actually sampling $p'_i$s? Commented Feb 4, 2015 at 17:53