I am trying to understand under what circumstances the log likelihood function of a point process concave. Assume that the process can be defined by a conditional intensity function and that the log likelihood function exists. Is there a general theory or good reference for understanding when the log likelihood function is concave (and hence you can reliably find the MLE)?


This depends on the parameterization of the conditional intensity function. The (regular) point process likelihood is given by,

$L = \left[ \prod_i \lambda^\ast(t_i) \right] \exp\left(-\int \lambda^\ast(u) \,\mathrm{d}u \right) $

with the conditional intensity function $\lambda^\ast(t)$ (from Daley & Vere-Jones, 2002). The main problem is that it is a functional; it is a function of realization history. Therefore, a finite or nonparametric parameterization is needed to practically estimate it.

A successful class of estimators assume a linear form of dependence from the history. In the neuroscience community, it is (unfortunately) known as GLM due to its resemblance to (Poisson) generalized linear models when time is discretized. Here the conditional likelihood is parametrized as $\lambda^\ast(t) = f(\mathbf{h}^\top \mathbf{r})$ where $\mathbf{r}$ is the discretized finite history of the process, $\mathbf{h}$ is the (finite vector) parameter, and $f(\cdot)$ is a (pointwise) nonlinear function. It can be shown that when $f$ is convex and log-concave, the log-likelihood is concave. See (Paninski 2004).

  • $\begingroup$ Thank you for this. Just to clarify, I am particularly interested in a point process defined by the conditional intensity $ \lambda(t) = \mu + \sum\limits_{t_i<t}{\alpha e^{-\beta(t-t_i)}}$ . How would this fit into your answer? $\endgroup$ – felix Aug 31 '14 at 19:15
  • $\begingroup$ @felix what is $\beta$? Is it a constant or a function? $\endgroup$ – Memming Aug 31 '14 at 20:40
  • $\begingroup$ @felix if your $\beta$ is a constant, then yes, it is concave. because exponential function is convex and log-concave (constant). $\endgroup$ – Memming Aug 31 '14 at 20:54
  • $\begingroup$ I should have said $\mu, \alpha, \beta > 0$ and $\beta > \alpha$. $\beta$ is not constant sadly. $\endgroup$ – felix Aug 31 '14 at 21:02
  • $\begingroup$ @felix I meant a scalar or a function. Sorry for my sloppy language. So, yes. You're log likelihood is concave. $\endgroup$ – Memming Aug 31 '14 at 21:08

The likelihood function, as a functional form, is also a density. Since we are interested in the logarithm of the likelihood, the issue can be broken in two:
a) Is the density "log-concave"? (is its logarithm a concave function of the random variable?)
b) Is the density (viewed as a function of the paramaters) log-concave in the parameters of interest?

The reason for this two-step approach, is that we have many established results regarding log-concavity of densities (for a list see, http://works.bepress.com/ted_bergstrom/31/).

Then, we can use various other results that provide conditions under which log-concavity is "inherited" when we examine the functional form with respect to the parameters (for example, when the parameters and the variable are linearly connected). In essence, these are "inheritance" rules for concavity/convexity in general.

To apply this approach to the much-more-to-the-point answer by @Memming, the log-likelihood in this case is

$$\ln L = \sum_i \ln \lambda^\ast(t_i) - \int \lambda^\ast(u) \,\mathrm{d}u $$

$$\Rightarrow \ln L = \sum_i \ln f(\mathbf{h}^\top \mathbf{r_i}) - \int f(u) \,\mathrm{d}u $$

If $f()$ is log-concave, then $\ln f()$ is concave in its argument, whatever that may be. Now, this argument is a linear combination of the elements of the parameter vector $\mathbf h$, so, again by established results, $\ln f()$ is also concave if viewed as a function of $\mathbf h$ alone. But then, the sum of concave functions is also concave.

Regarding the other component of the log-likelihood, the integral, we would want it convex so that its negative will be concave. A sufficient condition is that $f$ is non-decreasing, irrespective of whether it is concave or convex... I don't see how convexity of $f$ guarantees convexity of the integral, since a convex function can be decreasing in its argument. If on the other hand $f$ is non-decreasing due to other aspects of the whole setup, assuming convexity appears redundant.

I would welcome any explanation on this point.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.