18
$\begingroup$

The univariate exponential Hawkes process is a self-exciting point process with an event arrival rate of:

$ \lambda(t) = \mu + \sum\limits_{t_i<t}{\alpha e^{-\beta(t-t_i)}}$

where $ t_1,..t_n $ are the event arrival times.

The log likelihood function is

$ - t_n \mu + \frac{\alpha}{\beta} \sum{( e^{-\beta(t_n-t_i)}-1 )} + \sum\limits_{i<j}{\ln(\mu+\alpha e^{-\beta(t_j-t_i)})} $

which can be calculated recursively:

$ - t_n \mu + \frac{\alpha}{\beta} \sum{( e^{-\beta(t_n-t_i)}-1 )} + \sum{\ln(\mu+\alpha R(i))} $

$ R(i) = e^{-\beta(t_i-t_{i-1})} (1+R(i-1)) $

$ R(1) = 0 $

What numerical methods can I use to find the MLE? What is the simplest practical method to implement?

$\endgroup$
  • 2
    $\begingroup$ I have had success fitting $\mu$ and $\alpha$ by maximizing the MLE the LBFGS implementation in scipy. The log-likelihood is not concave in $\beta$ though, so I simply iterated over a range of $\beta$ values and picked the one with the maximum likelihood. Note that $\alpha < \beta$ is required for stationarity of the process. $\endgroup$ – Emaad Ahmed Manzoor Jan 5 '17 at 3:56
  • 1
    $\begingroup$ curious, what is the correct form of the λ(t) function using the values of R(i) instead of resumming at each step? $\endgroup$ – crow Sep 17 '17 at 21:53
7
$\begingroup$

The Nelder-Mead simplex algorithm seems to work well.. It is implemented in Java by the Apache Commons Math library at https://commons.apache.org/math/ . I've also written a paper about the Hawkes processes at Point Process Models for Multivariate High-Frequency Irregularly Spaced Data .

felix, using exp/log transforms seems to ensure positivity of the parameters. As for the small alpha thing, search the arxiv.org for a paper called "limit theorems for nearly unstable hawkes processes"

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Welcome to the site, @StephenCrowley. If you have your own question, please do not post it as ( / as part of) an answer. Click on the gray "ASK QUESTION" button at the top of the page & ask it there. If you have a question for clarification from the OP, you should ask it in a comment to the question post above. (Although frustratingly, you cannot do that until you reach 50 rep.) $\endgroup$ – gung - Reinstate Monica Jan 14 '13 at 18:58
3
$\begingroup$

I solved this problem using the nlopt library. I found a number of the methods converged quite quickly.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ I assume you're familiar with T. Ozaki (1979), Maximum likelihood estimation of Hawkes' self-exciting point processes, Ann. Inst. Statist. Math., vol. 31, no. 1, 145-155. $\endgroup$ – cardinal Sep 23 '12 at 15:26
  • 1
    $\begingroup$ Could you give more details of what you did? It seems there is a problem with setting constraints and also that large beta is indistinguishable from zero alpha (they both look Poisson). $\endgroup$ – felix Aug 14 '14 at 16:49
3
$\begingroup$

You could also do a simple maximization. In R:

neg.loglik <- function(params, data, opt=TRUE) {
  mu <- params[1]
  alpha <- params[2]
  beta <- params[3]
  t <- sort(data)
  r <- rep(0,length(t))
  for(i in 2:length(t)) {
    r[i] <- exp(-beta*(t[i]-t[i-1]))*(1+r[i-1])
  }
  loglik <- -tail(t,1)*mu
  loglik <- loglik+alpha/beta*sum(exp(-beta*(tail(t,1)-t))-1)
  loglik <- loglik+sum(log(mu+alpha*r))
  if(!opt) {
    return(list(negloglik=-loglik, mu=mu, alpha=alpha, beta=beta, t=t,
                r=r))
  }
  else {
    return(-loglik)
  }
}

# insert your values for (mu, alpha, beta) in par
# insert your times for data
opt <- optim(par=c(1,2,3), fn=neg.loglik, data=data)
| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ How do you ensure mu, alpha and beta are not set to negative values? $\endgroup$ – felix Aug 14 '14 at 8:50
  • $\begingroup$ You can set the lower and upper parameters in the optim call. $\endgroup$ – assumednormal Aug 14 '14 at 16:20
  • $\begingroup$ Not for Nelder-Mead you can't can you which is the default? (See stat.ethz.ch/R-manual/R-devel/library/stats/html/optim.html ) . Also, I don't think there is any way to distinguish huge beta from zero alpha is there so a general optimization seems doomed. $\endgroup$ – felix Aug 14 '14 at 16:44
1
$\begingroup$

Here is my solution to "What is the simplest practical method to implement?" using python, specifically numpy, scipy and tick.

One modification is that I set the exponential kernel such that alpha x beta x exp (-beta (t - ti)), to coincide with how tick defines exponential kernels: https://x-datainitiative.github.io/tick/modules/generated/tick.hawkes.HawkesExpKern.html#tick.hawkes.HawkesExpKern

I'm assuming the reader might not be familiar with python.

Import the relevant libraries:

import numpy as np
from scipy.optimize import minimize
from tick.hawkes import SimuHawkesExpKernels

Define the recursive function (R(i) in the question above) which returns an array the same size as the number of events:

def _recursive(timestamps, beta):
    r_array = np.zeros(len(timestamps))
    for i in range(1, len(timestamps)):
        r_array[i] = np.exp(-beta * (timestamps[i] - timestamps[i - 1])) * (1 + r_array[i - 1])
    return r_array

Define the log likelihood function specifying the various parameters:

def log_likelihood(timestamps, mu, alpha, beta, runtime):
    r = _recursive(timestamps, beta)
    return -runtime * mu + alpha * np.sum(np.exp(-beta * (runtime - timestamps)) - 1) + \
           np.sum(np.log(mu + alpha * beta * r))

Simulate some Hawkes data using the tick library (or some other means):

m = 0.5
a = 0.2
b = 0.3
rt = 1000

simu = SimuHawkesExpKernels([[a]], b, [m], rt, seed=0)
simu.simulate()
t = simu.timestamps[0]

Scipy's minimize function is probably the most common python optimisation for scalar functions. It expects a function with only two sets of parameters; those you want to minimise and those which are fixed. There are various possible minimisation methods, I am using the default which is L-BFGS-B for bounded problems and is a quasi-newton Method.

Note that I should have started with a brute search first but the question asked for the simplest practical method. I could have also split into a minimisation over mu and alpha and then used simulated annealing over beta since the log-logarithm is convex over mu and alpha only.

Define a new function to be used by the minimize function and returns the negative log-likelihood:

def crit(params, *args):
    mu, alpha, beta = params
    timestamps, runtime = args
    return -log_likelihood(timestamps, mu, alpha, beta, runtime)

Call the minimize function and set the bounds for m,a and b to be positive:

minimize(crit, [m + 0.1, a + 0.1, b+ 0.1], args=(t, rt), bounds=((1e-10, None), (1e-10, None), (1e-10, None),))

Which gives estimates of m,a,b: array([0.43835767, 0.25823306, 0.14769243])

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Although implementation is often mixed with substantive content in questions, we are supposed to be a site for providing information about statistics, machine learning, etc., not code. It can be good to provide code as well, but please elaborate your substantive answer in text for people who don't read this language well enough to recognize & extract the answer from the code. $\endgroup$ – gung - Reinstate Monica Jul 2 at 18:07
  • 1
    $\begingroup$ Hey @gung-ReinstateMonica thanks for the suggestion I have tried to put in some explanation please let me know what you think! $\endgroup$ – SuMau Jul 3 at 14:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.