I am missing something. I am trying to estimate an ARMA(2,2) model using Maximum Likelihood estimation via the scipy.optimize.minimie function.

I have simulated an ARMA(2,2) process via the statsmodel module with AR coefficients 0.75 and -0.25, and MA coefficients 0.35 and 0.5:

rparams = np.array([.75, -.25])
maparams = np.array([.35, .5])
ar = np.r_[1, -arparams] # add zero-lag and negate
ma = np.r_[1, maparams] # add zero-lag
y = arma_generate_sample(ar, ma, 1000)

When I estimate an ARMA(2,2) model using the statsmodels module I get reasonable close coefficients, so my simulation is correct.

But, when I try to estimate the coefficients using Maximum Likelihood and the scipy.optimize.minimize function I am not able to get any reasonalbe results. My code is:

def arma(vPar, vData):
    """ARMA log-likelihood assuming no constant""" 
    iT = len(vData)
    AR1 = vPar[0]
    AR2 = vPar[1]
    Var = vPar[2]**2
    vRes = np.zeros((iT, 1))
    vX = np.ones((iT, 1))*np.mean(vData)
    for t in range(2, iT):
        vX[t] = AR1*vX[t-1] + AR2*vX[t-2]
        vRes[t] = vData[t]-vX[t]
    Llk = -0.5*(iT*np.log(2*pi)+iT*np.log(Var)+np.sum(vRes**2)/Var)
    return Llk

x0 = np.array([.15, .11, .22])
res = minimize(arma, x0, args=(y), method='SLSQP', options={'disp': True})

This results in:

Iteration limit exceeded    (Exit mode 9)
            Current function value: nan
            Iterations: 101
            Function evaluations: 1496
            Gradient evaluations: 101

I've noticed that the minimizer sometimes inputs nan as parameter values, but I do now know why. What am I missing?

  • $\begingroup$ Your log likelihood is incorrect for ARMA. You LL only has AR no MA. $\endgroup$ – forecaster Jan 12 '17 at 1:33
  • $\begingroup$ You are certainly right. But that was actually on purpose as part of my debugging procedure, I just forgot to edit it back to the complete model before posting here. I'll edit the post asap. $\endgroup$ – mfvas Jan 12 '17 at 13:25

The reason why you NaN is because the parameter VAR in your likelihood function is in denominator and when the optimizer is trying to search the objective space it is using the value 0 sometimes and therefore causing NaN. Couple of work arounds: 1. Try to constrain your parameters in the optimizer to have a lower bound a small value such as 0.000001 OR 2. you can multiply parameter VAR by a small value of 0.000001 in the maximum likelihood function.

  • $\begingroup$ I already tried constraining the parameters to have a lower bound 0.0001. Maybe I did it wrong, but it didn't work. $\endgroup$ – mfvas Jan 11 '17 at 11:30
  • $\begingroup$ Can you show what you did in your post! $\endgroup$ – forecaster Jan 11 '17 at 11:47
  • 2
    $\begingroup$ @forecaster, what about suggesting a synonym structural-break to the tag structural-change? I would, but I do not have sufficient reputation. I think it is a relevant synonym, because if you search for "break" (thinking about structural breaks) it would be natural to get a suggestion structural-change, which is not the case now. You could also suggest it in Meta and give a link there pointing to where people can vote. Recently I did a similar thing with adf as a synonym to augmented-dickey-fuller, it worked nicely. $\endgroup$ – Richard Hardy Feb 9 '17 at 14:18
  • $\begingroup$ @RichardHardy I agree, structural-break is more commonly used term than structural-change. Is there a way that I can do this ? I tried to create a synonym but looks like structural-break should already be a tag. $\endgroup$ – forecaster Feb 9 '17 at 16:28
  • $\begingroup$ @forecaster, I don't know why that is the case. Maybe structural break used to be a tag that got burninated before. Or something like that. You could post a separate question on Meta if you have time. Or if not, I could do it, but I do not know all the details. $\endgroup$ – Richard Hardy Feb 9 '17 at 17:19

Here is a relevant link that you can have a look at:



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.