An unclear regularity condition for the multivariate case of the Cramér-Rao inequality

Question

I'm trying to make sense of the multivariate case of the Cramér-Rao inequality. One of its conditions of regularity states:

$$ \frac{\partial}{\partial\theta}\left[\int T(x)f\left(x;\theta\right)\space dx\right]=\int T(x)\left[\frac{\partial}{\partial\theta}f\left(x;\theta\right)\right]\space dx\hspace{10mm}(*) $$

here $\theta\in\mathbb{R}^d$ and $T(x)=\left(T_1(x),\dots,T_n(x)\right)\in\mathbb{R}^n$.

I don't understand equality $(*)$:

1. What kind of multiplication is performed inside the integrals?
2. What does it mean to integrate a function that does not take its values in $\mathbb{R}$, as do, presumably, the integrands on both sides?
3. What does the functional $\frac{\partial}{\partial\theta}$ mean on either side of the equality?

Any help will be appreciated.

Answer 1

1. $T(x)$ should be taken to be a column vector of $n$ real-valued statistics $\left[T_1(x),\dots,T_n(x)\right]^T$.
2. $\int T(x)f\left(x;\theta\right)\space dx$ should be interpreted as the column vector of length $n$ $$ \left[\int T_1(x)f\left(x;\theta\right)\space dx,\dots,\int T_n(x)f\left(x;\theta\right)\space dx\right]^T\hspace{10mm}(1) $$ Each component of this vector is a function from the $d$-dimensional set $\Theta\subseteq\mathbb{R}^d$ into $\mathbb{R}$.
3. The left side of $(*)$ should be interpreted as the Jacobian of the vector of functions $(1)$, i.e. denoting the components of vector $(1)$ by $g_1\left(\theta\right),\dots,g_n\left(\theta\right)$, respectively, the left side of $(*)$ is the following $n\times d$ matrix $$ \left[ \begin{array}{ccc} \frac{\partial g_1}{\partial\theta_1}\left(x;\theta\right) & \dots & \frac{\partial g_1}{\partial\theta_d}\left(x;\theta\right) \\ \vdots & \ddots & \vdots \\ \frac{\partial g_n}{\partial\theta_1}\left(x;\theta\right) & \dots & \frac{\partial g_n}{\partial\theta_d}\left(x;\theta\right) \end{array} \right] \hspace{10mm}(2)$$
4. Moving on to the right side of $(*)$, the expression $\frac{\partial}{\partial\theta}f\left(x;\theta\right)$ is the $d$ dimensional row vector $$ \left[\frac{\partial f}{\partial\theta_1}\left(x;\theta\right),\dots,\frac{\partial f}{\partial\theta_d}\left(x;\theta\right)\right]\hspace{10mm}(3) $$
5. The expression $$ T(x)\left[\frac{\partial}{\partial\theta}f\left(x;\theta\right)\right] $$ is the matrix multiplication of the $n\times1$ matrix $T(x)$ and the $1\times d$ matrix $(3)$, yielding the $n\times d$ matrix $$ \left[ \begin{array}{ccc} T_1(x)\frac{\partial f}{\partial\theta_1}\left(x;\theta\right) & \cdots & T_1(x)\frac{\partial f}{\partial\theta_d}\left(x;\theta\right)\\ \vdots & \ddots & \vdots \\ T_n(x)\frac{\partial f}{\partial\theta_1}\left(x;\theta\right) & \cdots & T_n(x)\frac{\partial f}{\partial\theta_d}\left(x;\theta\right) \end{array} \right] \hspace{10mm}(4) $$
6. Finally, the integral on the right of $(*)$ is the $n\times d$ matrix obtained from $(4)$ be integrating out the $x$ variable of each component. In other words, denoting each of the components of $(4)$ by $h_{i,j} \left(x;\theta\right)$ ($i\in \left\{1,\dots,n\right\}$, $j\in\left\{1,\dots,d\right\}$), $(4)$ is the matrix $$ \left[ \begin{array}{ccc} \int h_{1,1}\left(x;\theta\right)\space dx & \cdots & \int h_{1,d}\left(x;\theta\right)\space dx \\ \vdots & \ddots & \vdots \\ \int h_{n,1}\left(x;\theta\right)\space dx & \cdots & \int h_{n,d}\left(x;\theta\right)\space dx \end{array} \right] \hspace{10mm}(5) $$

In conclusion, combining $(2)$ and $(5)$, we see that $(*)$ boils down to the stipulation that for every $i\in\left\{1,\dots,n\right\}$ and every $j\in\left\{1,\dots,d\right\}$, the $j$th partial derivative of $T_i(x)f\left(x;\theta\right)$ w.r.t. $\theta$ can be carried under the integral sign, as follows $$ \frac{\partial}{\partial\theta_j}\int T_i(x)f\left(x;\theta\right)\space dx = \int T_i(x)\frac{\partial f}{\partial\theta_j}\left(x;\theta\right)\space dx $$