Does the approximation of the gradient of the log-likelihood imply the approximation of the log-likelihood? Let $ln(p(\mathcal{D};\theta))$ be the log-likelihood function, such that $\mathcal{D}=\{\mathbf{x}_1,\mathbf{x}_2,...,\mathbf{x}_N\}$ is the observed data and $\theta$ is a vector of parameters. Now suppose that:
$$
\frac{\partial \ ln(p(\mathcal{D};\theta))}{\partial \theta} \approx \frac{\partial g(\mathcal{D},\theta)}{\partial \theta}
$$
Where $g$ is a function that maps $\mathcal{D}$ and $\theta$ to a scalar in $\mathbb{R}$. Is it accurate to then say that:
$$
ln(p(\mathcal{D};\theta)) \approx g(\mathcal{D},\theta)
$$
 A: If we integrate this equation from some arbitrary $\theta_0$ to $\theta$, we get:
$$\frac{\partial \ln(p(\mathcal{D};\theta))}{\partial \theta} \approx \frac{\partial g(\mathcal{D},\theta)}{\partial \theta}\Rightarrow \ln p(\mathcal D;\theta) - \ln p(\mathcal D;\theta_0) \approx g(\mathcal D,\theta) - g(\mathcal D,\theta_0)$$
If we define $C=\ln p(\mathcal D;\theta_0)-g(\mathcal D,\theta_0)$, we have $\ln p(\mathcal D;\theta)\approx g(\mathcal D,\theta)+C$. That means that $\exp[g(\mathcal D,\theta)]$ is approximately proportional to the likelihood.
This is a very informal discussion, if we want to get more specific here we would need more information about the meaning of "$\approx$". Are the gradients asymptotically equal? Are they close in some region of the parametric space only? Can we describe this approximation in terms of big-O or something like that?
Anyway, I believe it's safe to say that $g(\mathcal D,\theta)$ is an approximation of $\ln p(\mathcal D, \theta)$, up to an additive constante.
