3
$\begingroup$

I am interested to know why running the R arima function's result has a different accuracy from my "manual" way of linear regression.

lh         # lh {datasets}

# Arima, Use "CSS" to get least squared.

arima(lh,c(1,0,0),method="CSS")$coef
#       ar1 intercept
# 0.5859943 2.4150521

# "manual" method
length(lh)        # 48

Y<-lh[2:48]
X<-lh[1:47]
lm(Y~X)$coeff
# (Intercept)           X 
#   0.9998652   0.5859870 

Of course, I know that the arima model should be interpreted as:

Y - 2.4150521 = 0.5859943 (X- 2.4150521)

and the "manual" method should be interpreted as:

Y = 0.5859870 X + 0.9998652

See: http://www.stat.pitt.edu/stoffer/tsa2/Rissues.htm

I am interested to know why does the slopes differ by a tiny amount, 0.5859870 and 0.5859943?

Question:

If the arima method is minimizing the same least squared sum as the "manual" method, how can I change the optimizing parameters (e.g. smaller error tolerance, or more iterations) to get the "manual" result?

If the arima method is not doing the same thing as the "manual" method, what is the arima function doing?

Any help would be greatly appreciated. Thank you.


An update after searching the stackExchange of other's question:

The following question is concerned about the forgetting to incluse the "CSS" term in the arima function to specify least-squares (instead of maximum likelihood)

Maximum Likelihood estimation for ARMA(1,1) in R

The following 2 URLs are concerned about the interpretation of the coefficients (which is explained in the stat.pitt.edu article)

https://stackoverflow.com/questions/36372603/verifying-arima-coefficients

https://stackoverflow.com/questions/27893735/arima-fails-to-estimate-a-simple-ar1-in-r

I have made sure that I interpreted the coefficients correctly in my original question, and remembered to use "CSS" to specify least-squared. I am still not sure why my "manual" method differs from the arima function in R (by a small amount/fraction).

Any help would be greatly appreciated. Thank you very much.

$\endgroup$
  • $\begingroup$ Have you tried looking up existing related questions? I think there has been something quite similar before. $\endgroup$ – Richard Hardy Jun 10 '17 at 15:56
  • $\begingroup$ Thanks for the reply. Some other similar questions that I have found are concerned about the interpretation of the coefficients, or the exclusion of the "CSS" to specify least square. I have updated my original post with what others have asked. Thank you. $\endgroup$ – CH Ben Jun 12 '17 at 2:39
  • $\begingroup$ duplicate: stats.stackexchange.com/questions/311177/… $\endgroup$ – Taylor May 10 '19 at 19:59
1
$\begingroup$

After looking at the documentation, I can modify the optimization control to set a lower tolerance for the convergence of the algorithm. The arima method uses "BFGS" or some kind of optimization algorithm.

options(digits=10)     # 10 decimal places

# Less accurate
arima(lh,c(1,0,0),method="CSS")$coef
#          ar1    intercept 
# 0.5859942753 2.4150521108 

# More accurate
arima(lh,c(1,0,0),method="CSS",optim.control=list(reltol=1e-16))$coef
#          ar1    intercept 
# 0.5859869717 2.4150572654 


# MANUAL METHOD
lm(lh[2:48]~lh[1:47])$coefficient
#  (Intercept)     lh[1:47] 
# 0.9998651719 0.5859869717 
$\endgroup$
1
$\begingroup$

The least squares problem for the AR(1) has a closed form solution. "lm" computes that solution. "arima" maximizes the likelihood numerically, so its solution will only be approximate here. This is why the estimates are closer to each other when you increase the precision of the solver.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.