I'm trying to make sense of the multivariate case of the Cramér-Rao inequality. One of its conditions of regularity states:

$$ \frac{\partial}{\partial\theta}\left[\int T(x)f\left(x;\theta\right)\space dx\right]=\int T(x)\left[\frac{\partial}{\partial\theta}f\left(x;\theta\right)\right]\space dx\hspace{10mm}(*) $$
here $\theta\in\mathbb{R}^d$ and $T(x)=\left(T_1(x),\dots,T_n(x)\right)\in\mathbb{R}^n$.

I don't understand equality $(*)$:

  1. What kind of multiplication is performed inside the integrals?
  2. What does it mean to integrate a function that does not take its values in $\mathbb{R}$, as do, presumably, the integrands on both sides?
  3. What does the functional $\frac{\partial}{\partial\theta}$ mean on either side of the equality?
Any help will be appreciated.

  • $\begingroup$ @Zen: Of course i know what a partial derivative is, but $\theta$ is not a variable, it is a shorthand for a list of $d$ variables, and in this case i'm not sure i understand what $\frac{\partial}{\partial\theta}$ means. $\endgroup$
    – Evan Aad
    Jun 15, 2013 at 15:19

1 Answer 1

  1. $T(x)$ should be taken to be a column vector of $n$ real-valued statistics $\left[T_1(x),\dots,T_n(x)\right]^T$.
  2. $\int T(x)f\left(x;\theta\right)\space dx$ should be interpreted as the column vector of length $n$ $$ \left[\int T_1(x)f\left(x;\theta\right)\space dx,\dots,\int T_n(x)f\left(x;\theta\right)\space dx\right]^T\hspace{10mm}(1) $$ Each component of this vector is a function from the $d$-dimensional set $\Theta\subseteq\mathbb{R}^d$ into $\mathbb{R}$.
  3. The left side of $(*)$ should be interpreted as the Jacobian of the vector of functions $(1)$, i.e. denoting the components of vector $(1)$ by $g_1\left(\theta\right),\dots,g_n\left(\theta\right)$, respectively, the left side of $(*)$ is the following $n\times d$ matrix $$ \left[ \begin{array}{ccc} \frac{\partial g_1}{\partial\theta_1}\left(x;\theta\right) & \dots & \frac{\partial g_1}{\partial\theta_d}\left(x;\theta\right) \\ \vdots & \ddots & \vdots \\ \frac{\partial g_n}{\partial\theta_1}\left(x;\theta\right) & \dots & \frac{\partial g_n}{\partial\theta_d}\left(x;\theta\right) \end{array} \right] \hspace{10mm}(2)$$
  4. Moving on to the right side of $(*)$, the expression $\frac{\partial}{\partial\theta}f\left(x;\theta\right)$ is the $d$ dimensional row vector $$ \left[\frac{\partial f}{\partial\theta_1}\left(x;\theta\right),\dots,\frac{\partial f}{\partial\theta_d}\left(x;\theta\right)\right]\hspace{10mm}(3) $$
  5. The expression $$ T(x)\left[\frac{\partial}{\partial\theta}f\left(x;\theta\right)\right] $$ is the matrix multiplication of the $n\times1$ matrix $T(x)$ and the $1\times d$ matrix $(3)$, yielding the $n\times d$ matrix $$ \left[ \begin{array}{ccc} T_1(x)\frac{\partial f}{\partial\theta_1}\left(x;\theta\right) & \cdots & T_1(x)\frac{\partial f}{\partial\theta_d}\left(x;\theta\right)\\ \vdots & \ddots & \vdots \\ T_n(x)\frac{\partial f}{\partial\theta_1}\left(x;\theta\right) & \cdots & T_n(x)\frac{\partial f}{\partial\theta_d}\left(x;\theta\right) \end{array} \right] \hspace{10mm}(4) $$
  6. Finally, the integral on the right of $(*)$ is the $n\times d$ matrix obtained from $(4)$ be integrating out the $x$ variable of each component. In other words, denoting each of the components of $(4)$ by $h_{i,j} \left(x;\theta\right)$ ($i\in \left\{1,\dots,n\right\}$, $j\in\left\{1,\dots,d\right\}$), $(4)$ is the matrix $$ \left[ \begin{array}{ccc} \int h_{1,1}\left(x;\theta\right)\space dx & \cdots & \int h_{1,d}\left(x;\theta\right)\space dx \\ \vdots & \ddots & \vdots \\ \int h_{n,1}\left(x;\theta\right)\space dx & \cdots & \int h_{n,d}\left(x;\theta\right)\space dx \end{array} \right] \hspace{10mm}(5) $$

In conclusion, combining $(2)$ and $(5)$, we see that $(*)$ boils down to the stipulation that for every $i\in\left\{1,\dots,n\right\}$ and every $j\in\left\{1,\dots,d\right\}$, the $j$th partial derivative of $T_i(x)f\left(x;\theta\right)$ w.r.t. $\theta$ can be carried under the integral sign, as follows $$ \frac{\partial}{\partial\theta_j}\int T_i(x)f\left(x;\theta\right)\space dx = \int T_i(x)\frac{\partial f}{\partial\theta_j}\left(x;\theta\right)\space dx $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.