Derivation of score vector

Question

Can anyone explain the process of this derivation, step by step? This derivation is from Joint Models for Longitudinal and Time-to Event Data by Dimitris Rizopoulos.

\begin{equation} \begin{aligned} S(\theta) &= \sum_i \frac{\partial}{\partial \theta^\top}\log\int p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(b_i;\theta)db_i \\ &= \sum_i\frac{1}{p(T_i, \delta_i, y_i; \theta)}\frac{\partial}{\partial \theta^\top}\int p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(b_i;\theta)db_i \\ & = \sum_i\frac{1}{p(T_i, \delta_i, y_i; \theta)}\int \frac{\partial}{\partial \theta^\top}\{ p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(b_i;\theta)\}db_i \\ & =\sum_i\int\bigg[ \frac{\partial}{\partial \theta^\top}\log\{ p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(b_i;\theta)\}\bigg]\\ & \times \frac{p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(b_i;\theta)}{p(T_i, \delta_i, y_i; \theta)}db_i \\ &=\sum_i\int A(\theta,b_i)p(b_i \mid T_i, \delta_i, y_i; \theta)db_i, \end{aligned} \end{equation}

where $A(\cdot)$ denotes the complete data score vector, given by $A(\theta,b_i) = \partial\{\log p(T_i, \delta_i \mid b_i; \theta) + \log p(y_i \mid b_i;\theta) + \log p(b_i;\theta)\}/\partial \theta^\top$.

I also do not understand using a posterior distribution for the random effects $b_i$, since this is a frequentist derivation. Bayesians would have no need to integrate over the random effects, since the random effects vector is considered a parameter in the Bayesian regime.

Answer 1

First recall that $(\log f)' = \frac{f'}{f}$.

We have

$$ \frac{\partial}{\partial \theta} \log\int p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(b_i;\theta)db_i = \frac{\frac{\partial}{\partial \theta} \int p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(b_i;\theta)db_i}{\int p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(b_i;\theta)db_i} $$

Now$^*$, \begin{align*} \int p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(b_i;\theta)db_i &= \int p(T_i,\delta_i,y_i \mid b_i ; \theta)p(b_i; \theta)db_i \\ &= p(T_i,\delta_i,y_i; \theta) \end{align*} since we integrate the conditional probability it gives the marginal probability.

Under some regularity/integrability asumptions we have,

$$ \frac{\partial}{\partial \theta}\int p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(b_i;\theta)db_i = \int \frac{\partial}{\partial \theta}\{ p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(b_i;\theta)\}db_i. $$

Since $(\log f)' = \frac{f'}{f} \Longrightarrow f'= f (\log f)'$

Thus we can rewrite, $$ \frac{1}{p(T_i, \delta_i, y_i; \theta)}\int \frac{\partial}{\partial \theta}\{ p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(b_i;\theta)\}db_i \\ $$ as: $$ \int \frac{\partial}{\partial \theta} \log \big( p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(b_i;\theta)\big) \times \frac{p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(b_i;\theta)}{p(T_i, \delta_i, y_i; \theta)} dbi_i $$

From the Bayes formula we have$^*$: \begin{align*} \frac{p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(b_i;\theta)}{p(T_i, \delta_i, y_i; \theta)} &= \frac{p(T_i, \delta_i,y_i \mid b_i; \theta)p(b_i;\theta)}{p(T_i, \delta_i, y_i; \theta)} \\ &=p(b_i \mid T_i,\delta_i,y_i ; \theta) \end{align*} Thus finally if we note $$ A(\theta,b_i) = \frac{\partial}{\partial \theta} \log \big( p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(b_i;\theta)\big) $$

We have, $$ \frac{\partial}{\partial \theta^\top}\log\int p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(b_i;\theta)db_i \\ = \int A(\theta,b_i) p(b_i \mid T_i,\delta_i,Y_i ; \theta) db_i $$

Thus sums follows by taking this respect to each individual in the sample.

$^*$ here we use $(T, \delta) \perp Y \mid b$ which I think is common assumption in joint modeling