I am creating an application that can take as inputs, two numbers (1 or 0) as well as a class defining a binary operation (AND OR XOR etc) and training the network to preform the operation.
Without teaching the network operations I have trained individual networks on individual operators (IE on simple AND with two booleans as input) with great success.
I am however having issues encoding my class. Currently I am encoding the feature vector with the two inputs (1 or 0) and then using booleans to encode the categorical variable (operation) as I have 6 operations, i use 3 booleans to encode which operation is on that feature (3 booleans can encode 2^3 features). I then train the network on this 5 length feature vector, but am having very little success.
The independent networks have MSE of < 10^-10, that is they always return the correct answer for their respective gates, however the combined network with operations has MSE of 0.3, which could just be lucky as many operations have similar outputs (0.5 chance of randomly getting correct output). This is currently the setup:
X_train = np.array(self.dictionary.preprocess(train_input, 2), dtype=np.float64); y_train = np.array(train_output, dtype=np.float64) self.regressor = skflow.TensorFlowDNNRegressor(hidden_units=[20,20], steps=train_steps, learning_rate=0.15, batch_size=1, verbose=0) self.regressor.fit(X_train, y_train)
dictionary.preprocess() simply replaces the strings with the boolean class representations. Ans insight as to why my results are so poor would be very helpful. Note that I have tried many different parameter tweaks (such as network topology, learning rate, etc) without significantly improving the results, which leads me to believe my theory that i can encode categorical features with boolean vectors may be incorrect.