TL;DR see the picture at the end

I am a complete beginner at machine learning, so sorry for any stupid mistakes.

I have an hourly time series where values depend mostly on the current month, weekday and hour. The goal is to predict it for one month in advance using a neural network (I have data from the past several years). I transform the data to look like this:

Value WeekdayMon WeekdayTue ... Hour0 Hour1 ... Hour23
123   0          1              1     0         0
456   0          1              0     1         0
0     0          1              0     0         0

As you can see, I use the value and the dummy variables for weekdays and hours as features. I also normalize the Value column by subtracting the mean and dividing by (max-min) (not shown in the table).

Then, I build the learning examples for my neural network. For example, if I want to forecast 30 days in advance by the previous 60 days' data, each of them looks like this:

X = [
  Value, WeekdayMon, WeekdayTue, ..., Hour0, Hour1, ..., Hour23,
  Value, WeekdayMon, WeekdayTue, ..., Hour0, Hour1, ..., Hour23,
  ...60*24 such sequences total, for each hour of previous 60 days...

Y = [Value, Value, Value, ..., Value] 30*24 such values, for each predicted hour

The neural network itself is a very simple feedforward network. I initialize it as follows with Keras library (Python):

import math
from keras.models import Sequential
from keras.layers import Dense

# 60 days, 24 hours each, 1+7+24 numbers for each hour (see the table above)
features_count = 60 * 24 * (1 + 7 + 24)
# Forecasting 30 days, 24 hours each
forecast_count = 30 * 24

# A heuristic I found somewhere
neurons_count = int(math.sqrt(features_count * forecast_count))

model = Sequential([
    Dense(neurons_count, activation='sigmoid', input_dim=features_count),
    Dense(forecast, activation='linear')

This forecast captures the overall pattern for days and even weekdays (e.g. the value is generally higher on Mondays, and this network predicts it successfully). However, it fails to predict the fluctations throughout a day. E.g., the predicted value usually stays low at 15:00, but is significantly higher at 12:00 and 18:00. This is not reflected in the forecast. Example plot for two days of a predicted month (don't ask why it's hand-drawn): plot

I suppose that I either need to change my features (storing 1+7+24 numbers for every hour seems kind of redundant to me), use different layers in a feedforward network (no idea here), or use a different network architecture (would recurrent network with LSTM work?). But, as I said, I am a total beginner and do not know what to start with.

What can I do to improve the quality of such forecast?

EDIT: I realize that having such huge amount of features is probably horribly wrong. I guess what I need is a concrete example of what a good feature set for my case could be. Currently I am trying out a much shorter feature set: weekday at the start of the forecast period; average value for the past 30 days, but the results are worse.

  • 1
    $\begingroup$ You might have too little data. WIth just 60 days to forecast 30 days ahead simple models would do better, and by simpler models I mean something else than a neural network. I have tried forecasting 1 to 5 periods ahead on 300 data points with a neural network and some other models, and neural network had by far the worst accuracy among all methods. $\endgroup$ – Richard Hardy Jun 25 '17 at 14:40
  • $\begingroup$ @RichardHardy I train my network on the data for several years, but only use a feature set of 60 days for every prediction. I mean, my sliding window which I use for generating training examples has a width of 60 days. Should I train on examples like [hourly data for 365 days] => [data for following 30 days]? $\endgroup$ – mlnewbie Jun 25 '17 at 14:44
  • 1
    $\begingroup$ I would use longer history. E.g. if the seasonal pattern is relatively stable over time, I would use all available data to extract the seasonal components. You do not have that many weekends over 60 days, so you might get better accuracy if you consider all the weekends you have. $\endgroup$ – Richard Hardy Jun 25 '17 at 14:46
  • $\begingroup$ @RichardHardy Are you suggesting to use longer history with my current features representation, or represent them in other way (I'm not sure if mine is ok or not)? It already barely fits in the RAM, and I feel like this problem is not that complicated to require streaming data. $\endgroup$ – mlnewbie Jun 25 '17 at 15:07
  • 1
    $\begingroup$ Scratch what I said before. I now see that you indeed have a whole lot of inputs, and your network likely has a problem figuring out how to use them properly. You could try getting rid of most of those inputs. You mainly need to represent seasonality (e.g. some dummies) and an autoregressive component (but 60 is likely way too much). $\endgroup$ – Richard Hardy Jun 25 '17 at 15:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.