# Forecast for ARIMA in R doesn't seem to fit

I've been trying to implement some ARIMA modelling of data in R (haven't been using R for long so not sure how well this is done), using the forecast library, but the forecasting part itself doesn't seem to work properly and I haven't been able to find why.

The function I use is roughly this :

arimamodel = function(valset, timeset, valtest, timetest)
{
linear      = lm(valset ~ timeset)
intercept   = linear$$coefficients[[1]] slope = linear$$coefficients[[2]]
lineardata  = (slope * timeset) + intercept
residue     = lineardata - valset

lineartest  = lm(valtest ~ timetest)
intercepttest   = lineartest$$coefficients[[1]] slopetest = lineartest$$coefficients[[2]]
lineardatatest  = (slopetest * timetest) + intercepttest
residuetest     = lineardatatest - valtest

partautocorr = pacf(residue, lag.max = 500, plot=TRUE)
lags = partautocorr$$lag[abs(partautocorr$$acf) > 0.1]
highpacf = partautocorr$$acf[lags] arorder = ifelse(length(lags[lags < 6])>0, max(lags[lags < 6]), 1) period = partautocorr$$acf[lags[lags > 5]]

arimamodel = Arima(residue, order=c(arorder,1,0), seasonal=list(order=c(0,1,1)), method="ML")
arimaforecast = forecast(object = arimamodel, h = length(valtest))
forecastdata = arimaforecast[["mean"]]
}


After removing the trends from both the training data and the test data and finding a rough order for the autoregression (some improvements to be done on that part), I train some ARIMA model on the data. The results aren't too bad, here's an example of the training data (black) vs. the ARIMA model (red)

On the other hand, the forecast part seems completely unrelated to those results. Here's the test data (black) vs. the ARIMA forecast (red) :

The plot doesn't seem related at all to even the ARIMA model on the training data. Am I misunderstanding how to use this library?

Edit : Here's some results using autoarima :

Training data :

Test data :

• Have you tried using different parameters and/or autoarima function from forecast package? I wouldn't expect that using just a single set of parameters, based on some kind of rule-of-thumb would work out-of-the-box... – Tim Dec 10 '18 at 9:11
• The results are slightly better using autoarima (cf edit), but the appearance of the forecast still seem very different from the fit on the training data itself. Is that normal? – Slereah Dec 10 '18 at 9:18
• Given that you have short and very noisy time-series..? Do you see any pattern in the data that you'd expect ARIMA to find? Notice that your test set seems to be completely different then training set, so I wouldn't expect the algorithm to give any meaningful results. – Tim Dec 10 '18 at 9:34