2
$\begingroup$

I got stuck in the derivation of the quasi-likelihood function. Namely, given an i.i.d sample $\{Y_i,X_i \}_{i=1}^n$ with $n$ the sample size, let the conditional mean and variance functions be specified as, respectively, $E(Y|X)=\mu(X;\beta)$ and $V(Y|X)=\alpha V(\mu(X;\beta))$ is a diagonal variance-covariance matrix, where $\alpha$ is a constant (e.g., $\mu(X_i;\beta)=X_i'\beta)$ and $\alpha V(\mu(X_i;\beta))=\alpha\exp(X_i'\beta)$).

Then, with a short-hand notation $\mu=\mu(X;\beta)$ and $\alpha V(\mu)=\alpha V(\mu(X;\beta))$, my question is how the following quasi-likelihood function is dervied?

$$Q(\mu,y)=\int_y^\mu \frac{y-t}{\alpha V(t)}dt.$$

I am aware of the fact that the above expression is a result of taking integral of a quasi-score function. Some existing posts, like

Quasi-likelihood/Quasi Poisson

did not discuss how the $Q(\mu,y)$ is derived. Any help is greatly appreciated in advance.

$\endgroup$
3
$\begingroup$

The idea is to show that if we standardize $Y$ then this function has all properties of a score function needed to develop asymptotic theory. Then, we can just treat the standardized function as the score process and work backward (by integrating) to get the"quasi-likelihood" since the score function is the derivative of the likelihood as you point out.

So with that logic in hand, all we need to do is check the desired properties of a score function. What properties are these?

  1. We need the expectation to be 0 under $\theta_0$ or in this case the true mean, $\mu$. i.e. we have found the maximize of the likelihood.
  2. The Fisher information gives us the variance of the score function, $Var(l'(X|\theta_0))=I(\theta_0)$. Where $I(\theta_0)=-\mathbb{E}[l''(X|\theta_0)]$, i.e. the expectation of the second derivative gives us the Fisher information.

So let us prove these properties hold.

Consider the following score standardizion of $Y$,

$$q = \frac{Y-\mu}{\alpha V(\mu)}$$

We will treat this random variable as our score function as we prove these properties.

$$\mathbb{E}[q]=\frac{\mathbb{E}[Y-\mu]}{\alpha V(\mu)}=0$$ $$Var(q) = \frac{\mathbb{E}[(Y-\mu)^2]}{\alpha^2 V(\mu)^2}=\frac{\alpha V(\mu)}{\alpha^2 V(\mu)^2}=\frac{1}{\alpha V(\mu)}$$ $$-\mathbb{E}[\frac{\partial q}{\partial \mu}]=-\mathbb{E}[\frac{\partial}{\partial \mu}V^{-1}(Y-\mu)/\alpha]=-\mathbb{E}[-V^{-1}/\alpha]=\frac{1}{\alpha V(\mu)}$$

So we have established that $q$ may be thought of as a score function. Now let us simply recover the quasi-likelihood for a single observation via integration,

$$Q(\mu|y)=\int_y^\mu \frac{y-t}{\alpha V(t)}dt$$

Finally, by independence, we may write the likelihood for the full data as,

$$Q(\mu|Y)=\sum_{i=1} Q(\mu_i|y_i)$$

So for some parametric distribution, we may apply this by determining what $\mu$ and $V(\mu)$ are and plugging them in, and integrating to get our quasi-likelihood. Finally, I recommend this book for all its uses and generalizations.

$\endgroup$
5
  • $\begingroup$ There was a slight mistake in the notation, hopefully, it is more clear now. Yes, we want to find out the likelihood of $\mu_i$ for the realized value $y_i$ so we integrate the "quasi-score" from the realized value to the mean, $\mu_i=E[Y_i]$. $\endgroup$
    – Ariel
    Jun 8 at 2:28
  • $\begingroup$ Thank you very much for such a thorough answer. What you have carefully explained above makes perfect sense to me. However, I guess my main question is how the $Q(\mu|Y_i)$ is derived? Can you elaborate more on how we can let us simply recover the quasi-likelihood for a single observation via integration? It seems to be a definite integral of q defined above, but why it integrates from $y$ to $\mu$? $\endgroup$
    – Rico
    Jun 8 at 2:46
  • $\begingroup$ Sorry but I still need more explanation here. If I understand you correctly, then the likelihood of $\mu_i$ conditioning on the realized value $y_i$ is in fact the conditional probability of $\mu_i$ given $Y_i=y_i$, or $P(y_i\leq \mu_i\leq \mu_i|Y_i=y_i)$? Or is there anything I misunderstand? $\endgroup$
    – Rico
    Jun 8 at 2:54
  • $\begingroup$ It's actually just from the Fundamental Theorem of Calculus. Consider, $$\frac{\partial}{\partial \mu}Q(\mu|Y)=\frac{\partial}{\partial \mu} \sum_{i=1}\int_{y_i}^\mu\frac{y_i-t}{\alpha V(t)}dt=\sum_{i=1}\frac{\partial}{\partial \mu} \int_{y_i}^\mu \frac{y_i-t}{\alpha V(t)}dt=\sum_{i=1}\frac{y_i-\mu}{\alpha V(\mu)}=Q(\mu|Y)$$ Where the second to last equality follows from the FTC. Thus, we see that by defining our quasi-likelihood in this way it follows the usual relationship of the score and the likelihood still hold. $\endgroup$
    – Ariel
    Jun 8 at 3:21
  • $\begingroup$ I got it! Thank you very much for the help! $\endgroup$
    – Rico
    Jun 8 at 4:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.