# Multi-output Neural Network only predicting one value

I have been using LSTM multi-output Neural Nets to perform two tasks, regression coupled with a classification. The data is in a time-series format where my dependent variable is trade quantity between nations as well as an indicator as to whether they trade at all. The problem I'm running into, is that my network predicts only a single value for the regression (~0.36), which is not the mean for the training data, and only 1's (100% probability for each sample) for the classification.

The normalised data distributions for training, validation and testing look like this

For the classification problem there is about a 5:1 ratio of 1's to 0's but nothing excessively imbalanced. My network setup looks like this, but I would usually pass it through a genetic algorithm for hyper parameter tuning.

callback = tf.keras.callbacks.EarlyStopping(monitor='loss', patience=3, min_delta=1e-9)

inputs = keras.layers.Input(shape=(x_train.shape[1], x_train.shape[2]))
layer = LSTM(128, input_shape=(x_train.shape[1], x_train.shape[2]), return_sequences=True)(inputs)
layer = keras.layers.Dropout(0.2)(layer)
layer = LSTM(64, return_sequences=False)(layer)
layer = keras.layers.Dropout(0.2)(layer)
layer = Dense(32, activation='relu')(layer)

partner_layer = Dense(1, activation='softmax', name='partner_pred')(layer)
trade_layer = Dense(1, name='trade_pred')(layer)  # No activation due to regression, possible sigmoid as values are between 0 and 1.


• Do you mean a predicted probability of $1$? Neural networks, like logistic regressions, output probability values, not categories. – Dave Jun 15 at 9:55
• It does start out decreasing, but only in the first epoch. I'm only interested in the regression part of the problem, yet without adding the classification, the network only predicts zeros. Meaning I would consider it predicting something $>0$ as some kind of success. This is why I don't think setting the classification loss to zero is of much use.The weights are chosen arbitrarily, but I thought that I need to give the classification some higher weight, as its loss is mostly between 0 and 1 unlike the loss for the regression. – SimonDude Jun 15 at 13:40