2
$\begingroup$

I am doing work on AIC comparisons. For this purpose, I am trying to understand how log-likelihood is calculated for exponential smoothing models (ETS models) in different R packages.

In particular, ets in R forecast and es in R smooth appear to give very different log-likelihood values for the same model fitted to the same data set, as follows.

# data simulation y as ETS(A,N,N)
set.seed(546)
n = 300
y <- smooth::sim.es("ANN", obs = n)

# fit E(A,N,N) and ARIMA(0,1,1) models to simulated data with R packages smooth and forecast
es_smooth_model <- smooth::es(y$data, model = "ANN", initial ="optimal")
es_forecast_model <- forecast::ets(y$data, model = "ANN", lambda = NULL)

Not surprisingly, both models give near identical fitted values, residuals, forecasts, coefficients, error variances, etc.

However, their log-likelihood estimates give very different values

# comparing loglikelihoods
es_smooth_model$logLik    # -1215.57
es_forecast_model$loglik  # -1645.456

Manual calculation for both models (assuming Gaussian error distributions) gives:

#manual loglikelihoods:

# es_smooth_model :         -1215.578
- 0.5 * (n * (log(2) + log(pi) + log(es_smooth_model$s2)) +
  1/ es_smooth_model$s2 * sum(es_smooth_model$residuals^2))

# es_smooth_model :         -1215.574
- 0.5 * (n * (log(2) + log(pi) + log(es_forecast_model$sigma2)) +
           1/ es_forecast_model$sigma2 * sum(es_forecast_model$residuals^2))

Can somebody help me to understand why forecast::ets gives a different log-likelihood value? And which of the two log-likelihoods is the correct value?

$\endgroup$

1 Answer 1

1
$\begingroup$

For this model, the log likelihood is given by $$-\frac{n}{2}\log(2\pi\sigma^2) - \frac12 \sum_{t=1}^n e_t^2 / \sigma^2.$$ Taking the partial derivative with respect to $\sigma^2$ and setting it to zero gives the MLE of $\sigma^2$ to be $$\hat{\sigma}^2 = \frac1n\sum_{t=1}^n e_t^2$$ Substituting $\hat\sigma^2$ for $\sigma^2$ in the log likelihood gives the concentrated log likelihood: $$-\frac{n}{2}\log\left(\frac{2\pi e}n\right) -\frac{n}{2} \log\left( \sum_{t=1}^n e_t^2\right) = c_n -\frac{n}{2} \log\left( \sum_{t=1}^n e_t^2\right), $$ where $c_n = -\frac{n}{2}\log(2\pi e/n)$, which does not depend on the data. The forecast package drops $c_n$ as it makes no difference to the parameter estimates, or the calculation of the AIC, and saves a small amount of computation.

Reference: Hyndman et al (2008, p69).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.