Parameter estimation for random variables where a control parameter is another r.v

Question

Let $\{X_i\}$ a sequence of independent random variables. Each $X_i$ has a p.d.f $p(m, \theta)$. Where $\theta$ is a real unknown parameter and $m$ the outcome of another random variable $M$ with p.d.f $p(m)$.

I have the next protocol to get a sample $\bar{x}=\{x_1,...,x_n\}$:

First, I get the value $m_1 \sim p(m)$. Then I obtain the value $x_1 \sim p(m_1 \theta)$. And I repeat the process $n$-times.

The aim is to estimate the value of $\theta$ from $\bar{x}$.

My questions:

1. the maximum likelihood estimator is the best way to do that?
2. Are there lower bound inequalities for the variance of the estimator for $\theta$?

When $m$ and $\theta$ are real unknown parameters, I know that there is the Cramér-Rao Bound, and when one has a parameter that is a random variable there exists the Van Trees inequality. But in this situation, I don't know if there is a standard inequality.

Answer 1

Regarding your questions:

1. I think there is a confusion: it does not really make sense to ask if the MLE is a "good way" to estimate $\theta$. As its name suggests, the MLE is your best estimate of $\theta$ that maximizes the likelihood of your observation, so yes, it is pretty good ^^ But a more relevant question is how to obtain your MLE. Given that you have both observed variables $X_i$ and hidden variables $M_i$, I would suggest using the Expectation-Maximization algorithm.
2. You can indeed use the Cramér-Rao bound. In this case, the likelihood used to compute the Fisher Information is the likelihood of your observations $X_i$, obtained by marginalizing the joint distribution of the observed and hidden variables:

$$ p(X_i|\theta) = \sum_{M_i}p(X_i|M_i,\theta)p(M_i) $$

I highly recommend this article, in which the authors use the EM algo to obtain the MLE of a vector of parameters $\theta$, and also compute the Fisher Information matrix: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4867027/

Answer 2

The (complete information) likelihood in this model is$$\prod_i p(x_i|m_i,\theta)\tag{1}$$which can be maximised or used in a Bayesian analysis, providing a prior $p(\theta)$ is chosen. (There is no "best way to estimate" a quantity, it all depends on the utility function for running the estimation.)

In the event the $m_i$'s are not observed, the (observed) likelihood becomes$$\prod_i \int p(x_i|m,\theta)\,p(m)\,\text{d}m\tag{2}$$

If there exists an unbiased estimator of $\theta$ in this setting, $T_n(X_1,\ldots,X_n|m_1,\ldots,m_n)$ then it satisfies the Cramér-Rao inequality$$\text{cov}(T_n(X_1,\ldots,X_n|m_1,\ldots,m_n)) \ge I_n^{-1}(\theta;m_1,\ldots,m_n)$$where $I_n(\theta;m_1,\ldots,m_n)$ is the (complete) Fisher information matrix associated with (1): $$I_n(\theta;m_1,\ldots,m_n)=-\sum_{i=1}^n\mathbb{E}_\theta \left[ \dfrac{\partial ^2}{\partial \theta \, \partial \theta^\text{T}} \log p\left(X| m_i,{\theta}\right)\right]$$ Once again, if the $m_i$'s are not observed, the observed Fisher information become $$I_n(\theta)=-n \mathbb{E}_\theta \left[ \dfrac{\partial ^2}{\partial \theta \, \partial \theta^\text{T}} \log p\left(X| {\theta}\right)\right]=-n \mathbb{E}_\theta \left[ \dfrac{\partial ^2}{\partial \theta \, \partial \theta^\text{T}} \log \int p\left(X| m,{\theta}\right)p(m)\,\text{d}m \right]$$