TL;DR see the picture at the end
I am a complete beginner at machine learning, so sorry for any stupid mistakes.
I have an hourly time series where values depend mostly on the current month, weekday and hour. The goal is to predict it for one month in advance using a neural network (I have data from the past several years). I transform the data to look like this:
Value WeekdayMon WeekdayTue ... Hour0 Hour1 ... Hour23 123 0 1 1 0 0 456 0 1 0 1 0 0 0 1 0 0 0
As you can see, I use the value and the dummy variables for weekdays and hours as features.
I also normalize the
Value column by subtracting the mean and dividing by
(max-min) (not shown in the table).
Then, I build the learning examples for my neural network. For example, if I want to forecast 30 days in advance by the previous 60 days' data, each of them looks like this:
X = [ Value, WeekdayMon, WeekdayTue, ..., Hour0, Hour1, ..., Hour23, Value, WeekdayMon, WeekdayTue, ..., Hour0, Hour1, ..., Hour23, ...60*24 such sequences total, for each hour of previous 60 days... ] Y = [Value, Value, Value, ..., Value] 30*24 such values, for each predicted hour
The neural network itself is a very simple feedforward network. I initialize it as follows with Keras library (Python):
import math from keras.models import Sequential from keras.layers import Dense # 60 days, 24 hours each, 1+7+24 numbers for each hour (see the table above) features_count = 60 * 24 * (1 + 7 + 24) # Forecasting 30 days, 24 hours each forecast_count = 30 * 24 # A heuristic I found somewhere neurons_count = int(math.sqrt(features_count * forecast_count)) model = Sequential([ Dense(neurons_count, activation='sigmoid', input_dim=features_count), Dense(forecast, activation='linear') ])
This forecast captures the overall pattern for days and even weekdays (e.g. the value is generally higher on Mondays, and this network predicts it successfully). However, it fails to predict the fluctations throughout a day. E.g., the predicted value usually stays low at 15:00, but is significantly higher at 12:00 and 18:00. This is not reflected in the forecast. Example plot for two days of a predicted month (don't ask why it's hand-drawn):
I suppose that I either need to change my features (storing 1+7+24 numbers for every hour seems kind of redundant to me), use different layers in a feedforward network (no idea here), or use a different network architecture (would recurrent network with LSTM work?). But, as I said, I am a total beginner and do not know what to start with.
What can I do to improve the quality of such forecast?
EDIT: I realize that having such huge amount of features is probably horribly wrong. I guess what I need is a concrete example of what a good feature set for my case could be. Currently I am trying out a much shorter feature set:
weekday at the start of the forecast period; average value for the past 30 days, but the results are worse.