# Using continuous features for RNN training

I am designing a recurrent neural network for binary classification problem: (1) there is an attack in the network, (2) the session is normal in the network.

To achieve this, I am using the Kyoto University dataset. Here's a sample data from it:

duration, service, src_bytes, dest_bytes, count, same_srv_rate, serror_rate, srv_serror_rate, dst_host_count, dst_host_srv_count, dst_host_same_src_port_rate, dst_host_serror_rate, dst_host_srv_serror_rate, flag, ids_detection, malware_detection, ashula_detection, label, src_ip_add, src_port_num, dst_ip_add, dst_port_num, start_time, protocol
-0.026718199145531595,4,-0.0017137615074428484,-0.0023086230344278144,-0.3656989628802213,0.9201603673125098,2.8316170813302053,1.6838464062500405,-0.8300894248587679,-0.7122843212362112,3.505362993154133,2.2092313846051757,1.9096507395538231,6,0,0,0,1,110661,0.7951296522328849,5230,-0.18795233710676376,0.9228794625927296,1
-0.026718199145531595,4,-0.0017137615074428484,-0.0023086230344278144,-0.3073821073997234,0.9201603673125098,2.8316170813302053,1.6838464062500405,-0.8300894248587679,-0.6877003684627419,3.505362993154133,2.2092313846051757,1.9096507395538231,6,0,0,0,1,182016,0.7920093464532119,5230,-0.18795233710676376,0.9228794625927296,


The label 1 pertains to a state that there's an attack in the network, while the label 0 pertains to a state that there's no attack in the network.

(1) My problem is the usage of the continuous data such as duration, serror_rate among others. I'm wondering if one-hot encoding is applicable in this instance or not.

(2) How should I use the continuous data for the RNN training? Other features, categorical or ordinal ones, can be one-hot encoded. But what about the continuous ones?

UPDATE August 16, 2017

As @shimao suggested, I performed decile binning on my data, that is, to bin the features in 10 to 100 deciles. I used pandas to do so:

for index in range(len(cols_to_std)):
df[cols_to_std[index]] = pd.qcut(df[cols_to_std[index]], 10, labels=False, duplicates='drop')


The df above is a pandas DataFrame containing 21 features and 1 label.

Sample results of training with decile-binned data:

[0] loss : 3416.749902650714, accuracy : 0.9140625
[1] loss : 4271.98881316185, accuracy : 0.96875
[2] loss : 2105.836363852024, accuracy : 1.0
[3] loss : 3483.10527408123, accuracy : 0.98046875
[4] loss : 1750.0128374248743, accuracy : 0.97265625
[5] loss : 1320.9579735696316, accuracy : 0.99609375
[6] loss : 3481.7440667152405, accuracy : 0.97265625
[7] loss : 2572.4160171300173, accuracy : 1.0
[8] loss : 2453.3563360869884, accuracy : 0.95703125
[9] loss : 1284.2558837980032, accuracy : 1.0


Case solved. Thanks, @shimao!

In general, continuous data does not need to be encoded. If some of your categorical data comes in one-hot form and some in continous form, it is adequate to simply concatenate everything together in your input. It may help to scale/normalize the continuous data appropriately if the numbers involved are very big or small.

Sometimes, it does help to bin the continuous data. For example, compute the 0th, 10th, 20th... 100th percentile for the continuous parameter, then classify each continuous value into one of ten bins, the first bin being examples between 0th and 10th percentile, the second being 10th to 20th percentile, etc. Apply one-hot encoding.

Usually between 10 and 100 bins is a good number. This may or may not work better than just passing in the continuous value -- it is impossible to know without actually trying it out.

Cases I'm aware of where binning continuous values is common : pose estimation, image generation, bounding box proposal.

Cases where binning is not as common : image classification, depth estimation, bounding box regression.

• My problem is sequence classification. From the sample I gave, each row has 23 features, and a label. I want to classify use each row for training the neural network, and of course, for testing as well. Will binning be okay with it? Aug 11 '17 at 0:27
• Yes, binning will be ok, even for time series days Aug 11 '17 at 2:44
• Thanks for your help! I did percentile binning as you suggested, and got my neural net train with nice results! :) Aug 16 '17 at 15:32
• I have a question. Is it okay to bin a standardized data? Sep 28 '17 at 9:39
• The sample data I wrote here is from a standardized dataset. Sep 28 '17 at 9:42