I have created an Artificial Neural Network with 4 categorical features and a binary outcome either 1 for suspicious or 0 for non-suspicious:
ParentPath ParentExe
0 C:\Program Files (x86)\Wireless AutoSwitch wrlssw.exe
1 C:\Program Files (x86)\Wireless AutoSwitch WrlsAutoSW.exs
2 C:\Program Files (x86)\Wireless AutoSwitch WrlsAutoSW.exs
3 C:\Windows\System32 svchost.exe
4 C:\Program Files (x86)\Wireless AutoSwitch WrlsAutoSW.exs
ChildPath ChildExe Suspicious
C:\Windows\System32 conhost.exe 0
C:\Program Files (x86)\Wireless AutoSwitch wrlssw.exe 0
C:\Program Files (x86)\Wireless AutoSwitch wrlssw.exe 0
C:\Program Files\Common Files OfficeC2RClient.exe 0
C:\Program Files (x86)\Wireless AutoSwitch wrlssw.exe 1
C:\Program Files (x86)\Wireless AutoSwitch wrlssw.exe 0
I have used sklearn for label encoding and one hot encoding on the data:
#Import the dataset
X = DBF2.iloc[:, 0:4].values
#X = DBF2[['ParentProcess', 'ChildProcess']]
y = DBF2.iloc[:, 4].values#.ravel()
#Encoding categorical data
from sklearn.preprocessing import LabelEncoder, OneHotEncoder
#Label Encode Parent Path
labelencoder_X_1 = LabelEncoder()
X[:, 0] = labelencoder_X_1.fit_transform(X[:, 0])
#Label Encode Parent Exe
labelencoder_X_2 = LabelEncoder()
X[:, 1] = labelencoder_X_2.fit_transform(X[:, 1])
#Label Encode Child Path
labelencoder_X_3 = LabelEncoder()
X[:, 2] = labelencoder_X_3.fit_transform(X[:, 2])
#Label Encode Child Exe
labelencoder_X_4 = LabelEncoder()
X[:, 3] = labelencoder_X_4.fit_transform(X[:, 3])
#Create dummy variables
onehotencoder = OneHotEncoder(categorical_features = [0,1,2,3])
X = onehotencoder.fit_transform(X)
I have split the data into a training and test set and run it on my gpu box with a nvidia 1080. I have tuned the hyperparameters and am now ready to use the model that is trained in a production environment with one test sample being tested at a time. Lets say I just want to test one sample:
ParentPath ParentExe ChildPath ChildExe
0 C:\Windows\Malicious badscipt.exe C:\Windows\System cmd.exe
The issue that I am running into is the training set has seen the ChildPath "C:\Windows\System" and the ChildExe "cmd.exe" which are normal, but the training set has not seen the ParentPath "C:\Windows\Malicous" or ParentExe "badscipt.exe" so these have not been label or one hot encoded. My big question is how to build a predictive model when not all of the categorical variables (in this case, file paths and file names) can be exhaustively enumerated?
I have seen examples using feature hashing but I'm not sure how to apply that or if that would even solve this problem. Any help or pointers would be greatly appreciated.