I want to get the second-order Taylor approximation for an expectation.

I have the following distribution, which is a Generalized Dirichlet distribution with parameters $\boldsymbol\alpha$ and $\boldsymbol\beta$.

\begin{equation} q(\boldsymbol\theta\mid\boldsymbol\alpha,\boldsymbol\beta)=\prod_i^n\frac{\Gamma(\alpha_{i} + \beta_{i})}{\Gamma(\alpha_i)\Gamma(\beta_i)} \theta_{i}^{\alpha_{i} -1} (1-\sum_{j=1}^{i}\theta_{j})^{\gamma_{i}} \end{equation}

\begin{equation} \gamma_i= \begin{cases} \beta_i - \alpha_{i+1} - \beta_{i+1}& \text{if } i\neq n\\ \beta_i-1 & \text{if } i= n \end{cases} \end{equation}

After bayesian inference, I want to estimate the parameters. I think that taylor series can help me to approximate this expectation: \begin{equation} \mathbb{E}_\theta\sim q\left[\frac{\theta_i}{\sum_i\theta_i\varphi_i}\right] \end{equation}

Second-order Taylor series is expressed as follows, where $H$ is the Hessian matrix and $Tr$ is the trace operator.

$$ \mathbb{E}\left[f(\boldsymbol\theta)\right] \approx f(\mathbb{E}\left[\boldsymbol\theta\right]) + \frac{1}{2} Tr\left(H(\mathbb{E}\left[\boldsymbol\theta\right]) Var(\boldsymbol\theta)\right) $$

However, because of that sole component ($\theta_i$), I don't know how to express $\frac{\theta_i}{\sum_i\theta_i\varphi_i}$ in terms of $\boldsymbol\theta$, so I will have $\mathbb{E}_\theta\sim q\left[f(\boldsymbol\theta)\right]$.

Any suggestions on this? or maybe a different approach?

  • $\begingroup$ If f is a function of $\theta_i$, then you replace all the random variable $\theta_i$, by mean of $\theta_i$, and then calculate f? $\endgroup$ – seanv507 Mar 23 at 6:45
  • $\begingroup$ 1) I tried making a change of variable. I thought that introducing the component $\theta_i$ in the numerator will give a GDD with different parameters. However, that change is not trivial since the parameters work as a chain (the previous and next one are used in $\gamma_i$). 2) I don't see how to express $f(\theta_i)$ can you elaborate a little more? 3) I was thinking in doing this $\frac{\theta_k \delta(k=i)}{\sum_k \theta_k\varphi_k}$, is it a valid expression of $f(\boldsymbol\theta)$ $\endgroup$ – c.uent Mar 23 at 19:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.