2
$\begingroup$

If Y1,Y2,...,Yn has distribution in exponential family depending on the parameters θ1,..., θp and Let L(θ; y) and l(θ; y) denote the likelihood and log-likelihood,

then how can I prove prove with the help of hint

Can I consider this enter image description here as a function of Y because if so then I can take integral of this times with pdf of Y1,Y2..Yn and then I can solve it from there.

But I am not sure if I can consider this as function of Y.

$\endgroup$

1 Answer 1

2
$\begingroup$

The fact that the score function has a null expectation at the true value of the parameter $$\mathbb E_\theta^\mathbf Y \underbrace{\left[\dfrac{\partial \ell(\theta;\mathbf Y)}{\partial\theta}\right]}_{\substack{\text{random variable}\\ \text{as function of $Y$}}}=0$$ is unrelated with exponential families. Writing $$\ell(\theta;\mathbf y)=\sum_{i=1}^n \log f(y_i;\theta)\tag{1}$$ and considering as a function of $\theta$ the gradient of (1) (in $\theta$) is \begin{align}\dfrac{\partial \ell(\theta;\mathbf y)}{\partial\theta} &= \sum_{i=1}^n \dfrac{\partial \log f(y_i;\theta)}{\partial\theta}\\ &=\sum_{i=1}^n \dfrac{1}{f(y_i;\theta)}\dfrac{\partial f(y_i;\theta)}{\partial\theta}\end{align} by standard (calculus) derivation rules. Now, if one transforms the random variable $Y_i$ by the mapping $$y \longmapsto \dfrac{1}{f(y;\theta)}\dfrac{\partial f(y;\theta)}{\partial\theta}$$ (which is indexed by $\theta$) one obtains another random variable $$\dfrac{1}{f(Y_i;\theta)}\dfrac{\partial f(Y_i;\theta)}{\partial\theta}$$ Then the expectation of this (transformed) random variable under its distribution (indexed by the same $\theta$) is \begin{align}\mathbb E_\theta^{Y_i}\left[\dfrac{1}{f(Y_i;\theta)}\dfrac{\partial f(Y_i;\theta)}{\partial\theta}\right]&=\int \dfrac{1}{f(y_i;\theta)}\dfrac{\partial f(y_i;\theta)}{\partial\theta} f(y_i;\theta) \,\text dy_i\\ &=\int \dfrac{\partial f(y_i;\theta)}{\partial\theta} \,\text dy_i\tag{2}\\ &= \dfrac{\partial}{\partial\theta} \int f(y_i;\theta) \,\text dy_i\tag{3}\\ &= \dfrac{\partial}{\partial\theta}\,1\\ &= 0 \end{align} assuming the likelihood $f(y_i;\theta)$ is sufficiently regular for (2) and (3) to be equal. (Which means it must be uniformly bounded by an integrable function, following Leibniz's rule.) This proof is actually available on Wikipedia.

$\endgroup$
2
  • $\begingroup$ ℓ(θ;y)=∑i=1nlogf(yi;θ) (1) , here you assumed that Y1,Y2 ,...Yn are independent but what if they are not independent? $\endgroup$
    – bluelagoon
    Nov 11, 2020 at 14:26
  • 1
    $\begingroup$ Then you have to decompose the likelihood according to the conditional densities. $\endgroup$
    – Xi'an
    Nov 11, 2020 at 17:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.