So me and a friend have been tasked to make a NeuralNet for a old Kaggle competition which is a regression problem as you have to predict the sales of the shop for upto x amount of months which is Here.
We've decided to use Keras and Pandas to preprocess the train dataset, The train dataset has the following fields
(The text inside parenthesis are NOT in the dataset but only to show you the range and data type)
Store(1-1115) DayOfWeek(1-7) Date(YYYY-MM-DD) Sales(int) Customers(int) Open(0 OR 1) Promo(0 OR 1) StateHoliday(0,a,b,c) SchoolHoliday(0 OR 1)
How We Process/Make The Datasets:
So we decided to delete 'Sales' from the trainning dataset and to use it as Y, the target which is where I come to the first question. Should we normalise the Y/Sales target data?
Now We process the rest of the trainning data as follows.
- We remove the date as we thought it would not provide much use especially if we put the data in order.
- We have changed State Holiday to a (0 or 1) binary value to state weather it is/isn't a State Holiday. We have also changed it to 0.25/0.5/0.75/1.0 to represent all the State Holidays, we did this in a different iteration of the dataset (the different iteration part is important later)
- We then normalised Customers to between 0-1 just like the sales
- We also removed School Holiday as we thought it didnt hold a strong enough value to the sales target
- We also removed Open as if the shop was shut it got no Sales and just didnt make sense to keep
- We also got rid of Day Of Week in One iteration of the dataset and kept it in another
Now my friend is much more of a mathematician than I am, I'm more of a Computer Scientist. So using another dataset which was also in the competition called Store.csv and it looks like the following:
Store(1-1115) StoreType(a-d) Assortment(a-c) CompetitionDistance(int Meters) CompetitionOpenSinceMonth(int M) CompetitionOpenSinceYear(int YYYY) Promo2(0 OR 1) Promo2SinceWeek(int) Promo2SinceYear(YYYY) PromoInterval(Str months seperated by csv)
Now my friend wrote a distance function to make N clusters for our trainning dataset, the function is as follows(We settled on 5 clusters):
I put the most similiar together based on customer distance store type assortment and promo
We then have 5 datasets which are split by our distance function.
We now 5 datasets and we remove the following:
- Sales - save this as Y/Target
So now our final dataset looks like the following:
DayOfWeek(1-7) Customers(normalised) Promo(0 OR 1) StateHoliday(0 OR 1)
So far when we train our the Neural Network the accuracy and validation accurcy of our network just converges to a number and does not budge, no matter what epoch or learning rates or momentum we set. We have tried differnt clusters that still converge to different numbers. We made one big dataset with all the clusters and the numebrs still converge.
We made a big datasete with no cluster seperation and the accuracy and validation accuracy just reaches a fixed number after 2/3/4 epochs and never changes.
the network we made looks like the following:
model = Sequential() model.add(Dense(8, input_dim=4, activation='relu')) model.add(Dropout(0.1)) model.add(Dense(3, activation='relu')) model.add(Dense(1, activation='relu')) # model.add(Dense(1, activation='linear')) model.compile(loss="MSE", optimizer="nadam", metrics=['accuracy']) model.fit(x=xt.as_matrix(), y=yt.as_matrix(), validation_data=(xv.as_matrix(), yv.as_matrix()), epochs=5000) print("stop")
We'v even tried adding removing a few features and the same problem still happens.
- Why is the numbers converging no matter it we change the dataset?
- Could this be a network shape/activation function/loss/optimizer issue?
- Is the problem with the dataset?
Is the clustering idea a good idea to pre process the data?
How would you process this data to get good results?
What type of layers/neurons would be appropriate?
Should we normalize the data?