# Time Series Forecast - Evaluating accuracy [closed]

I am very new to R and the forecast package authored by Rob Hyndman. I am working on a time series with 24 samples per hour. I trained a random forest regressor to forecast 6 hour ahead values and am using MAPE(Mean Absolute Percentage Error) on a held out duration as the accuracy metric.

I want to compare its accuracy with standard time series methods like ARMA and ARIMA models.

Time Series Sample

Here is what I have currently. data.csv has 576*16(16 days worth) samples and I wish to measure forecast accuracy on last 3 days.

library(forecast)
pv_ts <- ts(pv_data$V2) train_ts <- window(pv_ts, end=576*13) test_ts <- window(pv_ts, start=576*13+1) fit <- auto.arima(train_ts) accuracy(forecast(fit, h=576*3), test_ts) This gives me MAPE which is average of h = 1 to 576*3 samples ahead point forecast absolute errors. Question: How to find the average of h=144 ahead forecast absolute percent error of the estimates of samples in test_ts? Specifically, how to calculate$ \frac {\sum\limits_{T=576*13-h}^{576*16-h} \left\lvert \tfrac{\hat{e}_{T+h}}{y_{T+h}}\right\rvert }{576*3}$with h=144 where$\hat{e}_{T+h}=\hat{y}_{T+h | T}-y_{T+h}$? • Post the data... – Tom Reilly Aug 9 '16 at 13:20 • @TomReilly Created a gist. Please see the update. – SPV Aug 9 '16 at 13:28 • What is your question? – Richard Hardy Aug 9 '16 at 20:18 • @RichardHardy updated the question. – SPV Aug 9 '16 at 20:21 ## 1 Answer I'm not really sure what you're trying to achieve here. The forecast at h=144 is just a point estimate whereas the MAPE is a measure of average accuracy over a selection of point forecasts. As in your case, the sequence of point forcasts from h=1 to h=576*3is used to calculate the MAPE in your current example. If you want to evaluate the accuracy of your 6-hour ahead forecast, why not just look at the forecast error? I.e.$\hat{e}_{T+h}=\hat{y}_{T+h | T}-y_{T+h}$EDIT: Added a rough outline to a solution of what I think is your problem. You mention calculating the "average" for h=144 but the average is equal to the point estimate because you only have one forecast at this particular horizon. library(forecast) pv_data = read.csv("data.csv", header=FALSE) pv_ts <- ts(pv_data$V2)
fc <- forecast(fit, h=576*3)$mean # Store actual outcome y <- window(pv_ts, start=576*13+144, end=576*13+144) # Store point forecast for h=144 y_hat <- fc[144] # Calculate ape at h=144 ape <- abs((y_hat-y)/y) • I want to average h=144 ahead forecast absolute percent error of the estimates of samples in test_ts. Specifically calculate$ \frac {\sum\limits_{T=576*13-h}^{576*16-h} APE(\hat{e}_{T+h})}{576*3}$with h=144. I am not sure how to do that cleanly in R. – SPV Aug 9 '16 at 14:37 • I updated the question to make it clear. Please take a look. – SPV Aug 9 '16 at 14:47 • So you want to calculate the MAPE from h=1 to h=144 ? Or do you want to calculate$\left|\frac{(\hat{y}_{T+144 | T}-y_{T+144})}{y_{T+144}}\right|$? I'm still not sure what you're trying to achieve. – Billywob Aug 10 '16 at 6:45 • I want to calculate MAPE only for h=144 which is the average of$\left|\frac{(\hat{y}_{T+144 | T}-y_{T+144})}{y_{T+144}}\right|$– SPV Aug 10 '16 at 6:55 • Updated answer to what I think you want to do. However, like I mention, calculating the average of$\left|\frac{(\hat{y}_{T+144 | T}-y_{T+144})}{y_{T+144}}\right|\$ doesn't make any sense because this is just one number. – Billywob Aug 10 '16 at 7:38