Fitting an ARIMA model with conflicting indicators I have a time series data set to which I want to fit an ARIMA model.
In looking at the plot of the data, it seems stationary, albeit perhaps marginally so. No trends, no seasonal effects, somewhat constant variance, etc. But despite the marginality, it seems as though it could quite possibly be stationary.
When I plot the ACF and PACF, I get the clearest possible signs that I should try an AR(1) model. The ACF is sinusoidal and gradually tapers to 0, and the PACF has one significant value that shuts off after lag 1.
When I fit an ARIMA(1, 0 ,0) model, the residuals plots are all just what I would want, and the ar1 coefficient is significant.
This seems like a good model, and so normally I would think I might want to consider it.
The problem is that an augmented Dickey-Fuller test indicates the series is not stationary, despite the plot of the series looking reasonable and the rest of the outputs looking good.
In light of the non-stationary indicator, I tried a first differencing.
Now the plot of the first differenced data seems to be unquestionably stationary, and the augmented Dickey-Fuller test is consistent with that conclusion. But when I plot the ACF and PACF of the differenced data, I get no significant values at all. This is the signature of white noise.
Nonetheless, I decided to fool around with it. Among other things I tried an ARIMA(1, 1, 1) model. Not only do the residual plots look good, and not only are the two coefficients significant, but the AIC, AICc, and BIC are all lower in this model than in the ARIMA(1, 0, 0) model.
I feel as though there’s something I’m really missing here. How can a series that has ACF and PACF plots indicating white noise actually be fit with a good model?
And given these two conflicting indicators, what would be the wisest choice?
 A: The null of a unit root is not rejected in the ADF test and the null of stationarity in the KPSS test is not rejected either. This is an inconvenient situation since none of the hypotheses can be rejected.
In principle, as mentioned here, in this situation it may be more cautions to consider the presence of a unit root and take first differences to detrend the series. However, the remaining information that you give suggests that the data can be considered stationary. (The OP mentions that the estimate of the AR coefficient is lower than 0.9 and the ACF of the residuals looks like the ACF of white noise.)
This is probably a limiting case (close to unit root). In that case, taking first differences to the data will not make much harm or difference. If any, the forecasts of the ARIMA(0,1,0) will be flat (equal to the last observation), while the AR(1) will exhibit a smooth trend pattern.
It would also be interesting to check if the results of the tests are affected by the presence of some outlying observations or patterns (e.g., extreme value at some point or a shift in the level). In R, you can use the package
tsoutliers to check for this. 
If outliers are found, you could run again the ADF test on the series adjusted for the effect of those outliers.
