Let consider two observables, $x$ and $y$. Suppose that $y$ depends on the independent variable $x$ through the model $m(x; \boldsymbol{\theta})$, where $\boldsymbol{\theta}$ is a vector of model parameters. I want to estimate the Cramer-Rao bound of one of these parameters, for forecasting purposes. To this aim, I have to calculate the Fisher information matrix and inverse it. I should be able to compute it before doing any experiments.

Let assume that:

1. Measurements of $\{x;y\}$ are uncorrelated each other.
2. Measurements of the independent variable $x$ have no uncertainty.
3. Measurements of $y$ are Gaussian distributed, with fixed variance $\sigma^2$ (it's the uncertainty of each measurement of $y$).
4. The likelihood is Gaussian, i.e.: $L(\boldsymbol{\theta};\{x,y\}) = \dfrac{1}{\sqrt{|2 \pi C|}} e^{-\dfrac{1}{2} \left[y - m(x; \boldsymbol{\theta})\right]^{T} C^{-1} \left[y - m(x; \boldsymbol{\theta})\right]}$, where $C$ is the covariance matrix of the data.
5. $C$ does not depend on $\boldsymbol{\theta}$.

With these assumptions, the covariance matrix of the data is diagonal ($C = \dfrac{\mathbb{1}}{\sigma^2}$) and thus the elements of the Fisher information matrix are:

\begin{equation} F_{\alpha \beta} = \dfrac{1}{\sigma^2}\sum_{i}^{N} \dfrac{\partial m(x_i; \boldsymbol{\theta})}{\partial\theta_\alpha} \dfrac{\partial m(x_i; \boldsymbol{\theta})}{\partial\theta_\beta} \;, \end{equation}

where $N$ is the number of measurements.

Questions:

1. If so far it is correct, knowing $F$ implies knowing the measurements of $x_i$ a priori of the experiment; how can this be possible?
2. Can $F$ be computed and inverted analytically, in case of $N=1000$ measurements for example?
3. Even in case of $N = 1$, $F$ always has null determinant. In fact, $F$ is always of the type: \begin{pmatrix} a^2 & ab\\ ba & b^2 \end{pmatrix} How to deal with it?

Practical example:

My observables $\{x;y\}$ are respectively time and position $\{t;s\}$. The model is $m(t; \delta, \phi, \omega, G) = A e^{-\delta t} \cos(\phi - \sqrt{\omega^2 - \delta^2}t) + \dfrac{G}{\omega}$.

• In the case of $N=1$, $F$ has a null determinant.
• In the case of $N>1$, each element of $F$ is a sum over the values of $t$. The matrix cannot be computed without knowing the array of the measurements of $t$. Therefore, if you know the values of $t$, is the computation of the matrix can be done only numerically, do you agree?

At this point I can see only one possible way out: since I need to calculate the Cramer-rao bound of $G$, I could first marginalize over all the parameters; deleting all the columns and rows of $F$ related to the other parameters should do the trick (or this procedure should be applied to the inverse of F? I am not sure about this). As a result the reduced Fisher matrix is 1x1, that is trivially invertible. Does it make sense?

Any suggestion or example would be highly appreciated, if I'm missing something. Thank you in advance for your help.

Let consider two observables, $x$ and $y$. Suppose that $y$ depends on the independent variable $x$ through the model $m(x; \boldsymbol{\theta})$, where $\boldsymbol{\theta}$ is a vector of model parameters. I want to estimate the Cramer-Rao bound of one of these parameters, for forecasting purposes. To this aim, I have to calculate the Fisher information matrix and inverse it. I should be able to compute it before doing any experiments.

Let assume that

1. Measurements of $\{x;y\}$ are uncorrelated each other.
2. Measurements of the independent variable $x$ have no uncertainty.
3. Measurements of $y$ are Gaussian distributed, with fixed variance $\sigma^2$ (it's the uncertainty of each measurement of $y$).
4. The likelihood is Gaussian, i.e.: $L(\boldsymbol{\theta};\{x,y\}) = \dfrac{1}{\sqrt{|2 \pi C|}} e^{-\dfrac{1}{2} \left[y - m(x; \boldsymbol{\theta})\right]^{T} C^{-1} \left[y - m(x; \boldsymbol{\theta})\right]}$, where $C$ is the covariance matrix of the data.
5. $C$ does not depend on $\boldsymbol{\theta}$.

With these assumptions, the covariance matrix of the data is diagonal ($C = \dfrac{\mathbb{1}}{\sigma^2}$) and thus the elements of the Fisher information matrix are:

\begin{equation} F_{\alpha \beta} = \dfrac{1}{\sigma^2}\sum_{i}^{N} \dfrac{\partial m(x_i; \boldsymbol{\theta})}{\partial\theta_\alpha} \dfrac{\partial m(x_i; \boldsymbol{\theta})}{\partial\theta_\beta} \;, \end{equation}

where $N$ is the number of measurements.

Questions:

1. If so far it is correct, knowing $F$ implies knowing the measurements of $x_i$ a priori of the experiment; how can this be possible?
2. Can $F$ be computed and inverted analytically, in case of $N=1000$ measurements for example?
3. Even in case of $N = 1$, $F$ always has null determinant. In fact, $F$ is always of the type: \begin{pmatrix} a^2 & ab\\ ba & b^2 \end{pmatrix} How to deal with it?

Practical example:

My observables $\{x;y\}$ are respectively time and position $\{t;s\}$. The model is $m(t; \delta, \phi, \omega, G) = A e^{-\delta t} \cos(\phi - \sqrt{\omega^2 - \delta^2}t) + \dfrac{G}{\omega}$.

• In the case of $N=1$, $F$ has a null determinant.
• In the case of $N>1$, each element of $F$ is a sum over the values of $t$. The matrix cannot be computed without knowing the array of the measurements of $t$. Therefore, if you know the values of $t$, is the computation of the matrix can be done only numerically, do you agree?

At this point I can see only one possible way out: since I need to calculate the Cramer-rao bound of $G$, I could first marginalize over all the parameters; deleting all the columns and rows of $F$ related to the other parameters should do the trick (or this procedure should be applied to the inverse of F? I am not sure about this). As a result the reduced Fisher matrix is 1x1, that is trivially invertible. Does it make sense?

Any suggestion or example would be highly appreciated, if I'm missing something. Thank you in advance for your help.

Let consider two observables, $x$ and $y$. Suppose that $y$ depends on $x$ through the model $m(x, \boldsymbol{\theta})$, where $\boldsymbol{\theta}$ is a vector of model parameters. I want to estimate the Cramer-Rao bound of one of these parameters, for forecasting purposes. To this aim, I have to calculate the Fisher information matrix and inverse it. I should be able to compute it before doing any experiments.

Let assume that:

1. Measurements of $\{x;y\}$ are uncorrelated each other.
2. Measurements of $x$ have no uncertainty.
3. Measurements of $y$ are Gaussian distributed, with fixed variance $\sigma^2$ (it's the uncertainty of each measurement of $y$).
4. Estimators of the model parameters $\boldsymbol{\theta}$ are Gaussian distributed.
5. The likelihood is Gaussian, i.e.: $L(\boldsymbol{\theta};\{x,y\}) = \dfrac{1}{\sqrt{|2 \pi C|}} e^{-\dfrac{1}{2} \left[y - m(x, \boldsymbol{\theta})\right]^{T} C^{-1} \left[y - m(x, \boldsymbol{\theta})\right]}$, where $C$ is the covariance matrix of the data.
6. $C$ does not depends on $\boldsymbol{\theta}$.

With these assumptions, the covariance matrix of the data is diagonal ($C = \dfrac{\mathbb{1}}{\sigma^2}$) and thus the elements of the Fisher information matrix are:

\begin{equation} F_{\alpha \beta} = \dfrac{1}{\sigma^2}\sum_{i}^{N} \dfrac{\partial m(x_i, \boldsymbol{\theta})}{\partial\theta_\alpha} \dfrac{\partial m(x_i, \boldsymbol{\theta})}{\partial\theta_\beta} \;, \end{equation}

where $N$ is the number of measurements.

Questions:

1. If so far it is correct, knowing $F$ implies knowing the measurements of $x$ a priori of the experiment; how can this be possible?
2. Can $F$ be computed analytically for non-trivial functional forms of $m(x_i, \boldsymbol{\theta})$ (like the standard example in 1-dim with $m(\mu) = \mu$)?
3. Can $F$ be inverted analytically, if for example I have $N=1000$ measurements?

Any suggestion or example would be highly appreciated, if I'm missing something. Thank you in advance for your help.

Let consider two observables, $x$ and $y$. Suppose that $y$ depends on the independent variable $x$ through the model $m(x; \boldsymbol{\theta})$, where $\boldsymbol{\theta}$ is a vector of model parameters. I want to estimate the Cramer-Rao bound of one of these parameters, for forecasting purposes. To this aim, I have to calculate the Fisher information matrix and inverse it. I should be able to compute it before doing any experiments.

Let assume that:

1. Measurements of $\{x;y\}$ are uncorrelated each other.
2. Measurements of the independent variable $x$ have no uncertainty.
3. Measurements of $y$ are Gaussian distributed, with fixed variance $\sigma^2$ (it's the uncertainty of each measurement of $y$).
4. The likelihood is Gaussian, i.e.: $L(\boldsymbol{\theta};\{x,y\}) = \dfrac{1}{\sqrt{|2 \pi C|}} e^{-\dfrac{1}{2} \left[y - m(x; \boldsymbol{\theta})\right]^{T} C^{-1} \left[y - m(x; \boldsymbol{\theta})\right]}$, where $C$ is the covariance matrix of the data.
5. $C$ does not depend on $\boldsymbol{\theta}$.

With these assumptions, the covariance matrix of the data is diagonal ($C = \dfrac{\mathbb{1}}{\sigma^2}$) and thus the elements of the Fisher information matrix are:

\begin{equation} F_{\alpha \beta} = \dfrac{1}{\sigma^2}\sum_{i}^{N} \dfrac{\partial m(x_i; \boldsymbol{\theta})}{\partial\theta_\alpha} \dfrac{\partial m(x_i; \boldsymbol{\theta})}{\partial\theta_\beta} \;, \end{equation}

where $N$ is the number of measurements.

Questions:

1. If so far it is correct, knowing $F$ implies knowing the measurements of $x_i$ a priori of the experiment; how can this be possible?
2. Can $F$ be computed and inverted analytically, in case of $N=1000$ measurements for example?
3. Even in case of $N = 1$, $F$ always has null determinant. In fact, $F$ is always of the type: \begin{pmatrix} a^2 & ab\\ ba & b^2 \end{pmatrix} How to deal with it?

Practical example:

My observables $\{x;y\}$ are respectively time and position $\{t;s\}$. The model is $m(t; \delta, \phi, \omega, G) = A e^{-\delta t} \cos(\phi - \sqrt{\omega^2 - \delta^2}t) + \dfrac{G}{\omega}$.

• In the case of $N=1$, $F$ has a null determinant.
• In the case of $N>1$, each element of $F$ is a sum over the values of $t$. The matrix cannot be computed without knowing the array of the measurements of $t$. Therefore, if you know the values of $t$, is the computation of the matrix can be done only numerically, do you agree?

At this point I can see only one possible way out: since I need to calculate the Cramer-rao bound of $G$, I could first marginalize over all the parameters; deleting all the columns and rows of $F$ related to the other parameters should do the trick (or this procedure should be applied to the inverse of F? I am not sure about this). As a result the reduced Fisher matrix is 1x1, that is trivially invertible. Does it make sense?

Any suggestion or example would be highly appreciated, if I'm missing something. Thank you in advance for your help.

Let consider two variables, $x$ and $y$. Suppose that $y$ depends on $x$ through the model $m(x, \boldsymbol{\theta})$, where $\boldsymbol{\theta}$ is a vector of model parameters. I want to estimate the Cramer-Rao bound of one of these parameters, for forecasting purposes. To this aim, I have to calculate the Fisher information matrix and inverse it. I should be able to compute it before doing any experiments.

Let assume that:

1. Measurements of $\{x;y\}$ are uncorrelated each other.
2. Measurements of $x$ have no uncertainty.
3. Measurements of $y$ are Gaussian distributed, with fixed variance $\sigma^2$.
4. Estimators of the model parameters $\boldsymbol{\theta}$ are Gaussian distributed.
5. The likelihood is Gaussian, i.e.: $L(\boldsymbol{\theta};\{x,y\}) = \dfrac{1}{\sqrt{|2 \pi C|}} e^{-\dfrac{1}{2} \left[y - m(x, \boldsymbol{\theta})\right]^{T} C^{-1} \left[y - m(x, \boldsymbol{\theta})\right]}$, where $C$ is the covariance matrix of the data.
6. $C$ does not depends on $\boldsymbol{\theta}$.

With these assumptions, the covariance matrix of the data is diagonal ($C = \dfrac{\mathbb{1}}{\sigma^2}$) and thus the elements of the Fisher information matrix are:

\begin{equation} F_{\alpha \beta} = \dfrac{1}{\sigma^2}\sum_{i}^{N} \dfrac{\partial m(x_i, \boldsymbol{\theta})}{\partial\theta_\alpha} \dfrac{\partial m(x_i, \boldsymbol{\theta})}{\partial\theta_\beta} \;, \end{equation}

where $N$ is the number of measurements.

If so far it is correct, knowing $F$ implies knowing the measurements of $x$ a priori.

Therefore, I am wondering:

1. Can $F$ be computed analytically for non-trivial functional forms of $m(x_i, \boldsymbol{\theta})$ (like the standard example in 1-dim with $m(\mu) = \mu$)?
2. Can $F$ be inverted analytically, if for example I have $N=1000$ measurements?

Any suggestion or example would be highly appreciated, if I'm missing something. Thank you in advance for your help.

Let consider two observables, $x$ and $y$. Suppose that $y$ depends on $x$ through the model $m(x, \boldsymbol{\theta})$, where $\boldsymbol{\theta}$ is a vector of model parameters. I want to estimate the Cramer-Rao bound of one of these parameters, for forecasting purposes. To this aim, I have to calculate the Fisher information matrix and inverse it. I should be able to compute it before doing any experiments.

Let assume that:

1. Measurements of $\{x;y\}$ are uncorrelated each other.
2. Measurements of $x$ have no uncertainty.
3. Measurements of $y$ are Gaussian distributed, with fixed variance $\sigma^2$ (it's the uncertainty of each measurement of $y$).
4. Estimators of the model parameters $\boldsymbol{\theta}$ are Gaussian distributed.
5. The likelihood is Gaussian, i.e.: $L(\boldsymbol{\theta};\{x,y\}) = \dfrac{1}{\sqrt{|2 \pi C|}} e^{-\dfrac{1}{2} \left[y - m(x, \boldsymbol{\theta})\right]^{T} C^{-1} \left[y - m(x, \boldsymbol{\theta})\right]}$, where $C$ is the covariance matrix of the data.
6. $C$ does not depends on $\boldsymbol{\theta}$.

With these assumptions, the covariance matrix of the data is diagonal ($C = \dfrac{\mathbb{1}}{\sigma^2}$) and thus the elements of the Fisher information matrix are:

\begin{equation} F_{\alpha \beta} = \dfrac{1}{\sigma^2}\sum_{i}^{N} \dfrac{\partial m(x_i, \boldsymbol{\theta})}{\partial\theta_\alpha} \dfrac{\partial m(x_i, \boldsymbol{\theta})}{\partial\theta_\beta} \;, \end{equation}

where $N$ is the number of measurements.

Questions:

1. If so far it is correct, knowing $F$ implies knowing the measurements of $x$ a priori of the experiment; how can this be possible?
2. Can $F$ be computed analytically for non-trivial functional forms of $m(x_i, \boldsymbol{\theta})$ (like the standard example in 1-dim with $m(\mu) = \mu$)?
3. Can $F$ be inverted analytically, if for example I have $N=1000$ measurements?

Any suggestion or example would be highly appreciated, if I'm missing something. Thank you in advance for your help.