An unclear regularity condition for the multivariate case of the Cramér-Rao inequality 
I'm trying to make sense of the multivariate case of the Cramér-Rao inequality. One of its conditions of regularity states:


$$
\frac{\partial}{\partial\theta}\left[\int T(x)f\left(x;\theta\right)\space dx\right]=\int T(x)\left[\frac{\partial}{\partial\theta}f\left(x;\theta\right)\right]\space dx\hspace{10mm}(*)
$$


here $\theta\in\mathbb{R}^d$ and $T(x)=\left(T_1(x),\dots,T_n(x)\right)\in\mathbb{R}^n$.



I don't understand equality $(*)$:


*


*
What kind of multiplication is performed inside the integrals?



*
What does it mean to integrate a function that does not take its values in $\mathbb{R}$, as do, presumably, the integrands on both sides?



*
What does the functional $\frac{\partial}{\partial\theta}$ mean on either side of the equality?



Any help will be appreciated.

 A: *

*
$T(x)$ should be taken to be a column vector of $n$ real-valued statistics $\left[T_1(x),\dots,T_n(x)\right]^T$.



*
$\int T(x)f\left(x;\theta\right)\space dx$ should be interpreted as the column vector of length $n$

$$
\left[\int T_1(x)f\left(x;\theta\right)\space dx,\dots,\int T_n(x)f\left(x;\theta\right)\space dx\right]^T\hspace{10mm}(1)
$$

Each component of this vector is a function from the $d$-dimensional set $\Theta\subseteq\mathbb{R}^d$ into $\mathbb{R}$.



*
The left side of $(*)$ should be interpreted as the Jacobian of the vector of functions $(1)$, i.e. denoting the components of vector $(1)$ by $g_1\left(\theta\right),\dots,g_n\left(\theta\right)$, respectively, the left side of $(*)$ is the following $n\times d$ matrix

$$
\left[
\begin{array}{ccc}
\frac{\partial g_1}{\partial\theta_1}\left(x;\theta\right) & \dots & \frac{\partial g_1}{\partial\theta_d}\left(x;\theta\right) \\
\vdots & \ddots & \vdots \\
\frac{\partial g_n}{\partial\theta_1}\left(x;\theta\right) & \dots & \frac{\partial g_n}{\partial\theta_d}\left(x;\theta\right)
\end{array}
\right]
\hspace{10mm}(2)$$



*
Moving on to the right side of $(*)$, the expression $\frac{\partial}{\partial\theta}f\left(x;\theta\right)$ is the $d$ dimensional row vector

$$
\left[\frac{\partial f}{\partial\theta_1}\left(x;\theta\right),\dots,\frac{\partial f}{\partial\theta_d}\left(x;\theta\right)\right]\hspace{10mm}(3)
$$



*
The expression

$$
T(x)\left[\frac{\partial}{\partial\theta}f\left(x;\theta\right)\right]
$$

is the matrix multiplication of the $n\times1$ matrix $T(x)$ and the $1\times d$ matrix $(3)$, yielding the $n\times d$ matrix

$$
\left[
\begin{array}{ccc}
T_1(x)\frac{\partial f}{\partial\theta_1}\left(x;\theta\right) & \cdots & T_1(x)\frac{\partial f}{\partial\theta_d}\left(x;\theta\right)\\
\vdots & \ddots & \vdots \\
T_n(x)\frac{\partial f}{\partial\theta_1}\left(x;\theta\right) & \cdots & T_n(x)\frac{\partial f}{\partial\theta_d}\left(x;\theta\right)
\end{array}
\right]
\hspace{10mm}(4)
$$



*
Finally, the integral on the right of $(*)$ is the $n\times d$ matrix obtained from $(4)$ be integrating out the $x$ variable of each component. In other words, denoting each of the components of $(4)$ by $h_{i,j}
\left(x;\theta\right)$ ($i\in \left\{1,\dots,n\right\}$, $j\in\left\{1,\dots,d\right\}$), $(4)$ is the matrix

$$
\left[
\begin{array}{ccc}
\int h_{1,1}\left(x;\theta\right)\space dx & \cdots & \int h_{1,d}\left(x;\theta\right)\space dx \\
\vdots & \ddots & \vdots \\
\int h_{n,1}\left(x;\theta\right)\space dx & \cdots & \int h_{n,d}\left(x;\theta\right)\space dx
\end{array}
\right]
\hspace{10mm}(5)
$$




In conclusion, combining $(2)$ and $(5)$, we see that $(*)$ boils down to the stipulation that for every $i\in\left\{1,\dots,n\right\}$ and every $j\in\left\{1,\dots,d\right\}$, the $j$th partial derivative of $T_i(x)f\left(x;\theta\right)$ w.r.t. $\theta$ can be carried under the integral sign, as follows

$$
\frac{\partial}{\partial\theta_j}\int T_i(x)f\left(x;\theta\right)\space dx = \int T_i(x)\frac{\partial f}{\partial\theta_j}\left(x;\theta\right)\space dx
$$

