Suppose I have data $(y_i,x_i)_{i=1}^n$ and I know two density functions $f(y_i;\theta)$ and $g(x_i;\theta)$ -- two density function shares the same parameter $\theta$.

I want to use both information to estimate $\theta$. I consider log likelihood function $$L(\theta;data) = \sum_{i}\log(f(y_i;\theta)) + \log(g(x_i;\theta)).$$

if I maximize this with respect to $\theta$, would this give me a consistent estimator for $\theta$?


1 Answer 1


Sure, you can do this. It depends on how you think $y$ and $x$ are related. I'll give you an example in which this makes mathematical sense, but is contrived and unrealistic.

If $x$ and $y$ are independent but share a parameter. For instance, suppose $x_i \sim N(1, \theta)$ and $y_i \sim N(-1, \theta)$. Let $x = (x_1,x_2, \ldots, x_n)$ be an iid sample of size from $N(1, \theta)$ and $y = (y_1,y_2, \ldots, y_n)$ be an iid sample of size from $N(-1, \theta)$. Let $\mathcal{L}_x(\theta \mid x)$ be the likelihood function for $x$ and $\ell(\theta \mid x)$ be the log-likelihood function for $x$. We adopt similar notation for $y$.

The likelihood is then

\begin{align} \mathcal{L}(\theta \mid x, y)&= \left\{ \prod_{i = 1}^n \mathcal{L}_x(\theta \mid x_i) \right\} \times \left\{ \prod_{i=1}^n \mathcal{L}_y(\theta \mid y_i)\right\}\\ &=\prod_{i=1}^n \mathcal{L}_x(\theta \mid x_i) \mathcal{L}_y(\theta \mid y_i). \end{align} Using standard properties of logarithms gives us

\begin{align} \ell(\theta \mid x, y) &= \log \mathcal{L}(\theta \mid x, y)\\ &= \log \left\{ \prod_{i=1}^n \mathcal{L}_x(\theta \mid x_i) \mathcal{L}_y(\theta \mid y_i) \right\}\\ &= \sum_{i=1}^n \log \left\{\mathcal{L}_x(\theta \mid x_i) \mathcal{L}_y(\theta \mid y_i) \right\} \\ &= \sum_{i=1}^n \left\{ \log \left[\mathcal{L}_x(\theta \mid x_i)\right] + \left[ \mathcal{L}_y(\theta \mid y_i) \right] \right\}\\ &=\sum_{i=1}^n \left\{ \ell(\theta \mid x_i) + \ell(\theta \mid y_i) \right\} \end{align} In this case I imagine the estimator would be consistent under the usual regularity conditions. That's because we can break the sum up into two sums.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.