Why my validation loss is not converging over multistream model? [closed]

I want to merge two CNNs that are trained over the different dataset. I have taken two sequential models and merged them. But when using customized fit_generator, validation loss is not converging. How will I pass generators of different datasets?

input1_1 = keras.layers.Input(shape=(129,129,3))
x1 = keras.layers.Conv2D(kernel_size = (3,3), filters = 32,
activation='PReLU')(input1_1)

x3 = keras.layers.MaxPooling2D(2,2)(x1)
x4 = keras.layers.Conv2D(kernel_size = (5,5), filters = 64,
activation='relu')(x3)
x5 = keras.layers.MaxPooling2D(2,2)(x4)
x6 = keras.layers.Conv2D(kernel_size = (7,7), filters = 128,
activation='relu')(x5)
d1_1 = keras.layers.Dropout(0.5)(x6)
br1_1= keras.layers.MaxPooling2D(2,2)(d1_1)
br1_1 = keras.layers.Flatten()(br1_1)

input2_2 = keras.layers.Input(shape=(129,129,3))
x1 = keras.layers.Conv2D(kernel_size = (3,3), filters = 32,
activation='PReLU')(input2_2)

x3 = keras.layers.MaxPooling2D(2,2)(x1)
x4 = keras.layers.Conv2D(kernel_size = (5,5), filters = 64,
activation='relu')(x3)
x5 = keras.layers.MaxPooling2D(2,2)(x4)
x6 = keras.layers.Conv2D(kernel_size = (7,7), filters = 128,
activation='relu')(x5)
d2_2 = keras.layers.Dropout(0.5)(x6)
br2_2= keras.layers.MaxPooling2D(2,2)(d2_2)
br2_2 = keras.layers.Flatten()(br2_2)

# d2_4 = keras.layers.Dropout(0.4)(d2_3)
out1_1 = keras.layers.Dense(159,activation='softmax',kernel_regularizer=regularizers.l2(0.01),
activity_regularizer=regularizers.l1(0.01))(d2_3)
# model=keras.layers.Conv2DTranspose(kernel_size= (4,4), filters=10, activation='relu')(out)
modal1_1 = keras.models.Model(inputs=[input1_1,input2_2], outputs=out1_1)
modal1_1.summary()


I have tried to zip the generators(train_face, train_iris), but it is not doing the task.

def custom_iterator(Xp, Xs):

from keras.preprocessing.image import ImageDataGenerator

ig1 = ImageDataGenerator(rescale=1./255)
ig2 = ImageDataGenerator(rescale=1./255)
temp1 = ig1.flow_from_directory(Xp,target_size = (129, 129),batch_size =
10,class_mode = "categorical")
temp2 = ig2.flow_from_directory(Xs,target_size = (129, 129),batch_size =
10,class_mode = "categorical")

for batch in zip(temp1,temp2):

yield [batch[0][0], batch[1][0]], [batch[0][1]]


After performing Zip if i call fit generator my validation loss is not converging.

train_data_dir = "C:\\Users\\Desktop\\SDUMLA\\faceDataset\\mixTrain"
validation_data_dir = "C:\\Users\\Desktop\\SDUMLA\\faceDataset\\mixTest"
train_data_dir_nir = "C:\\Users\\Desktop\\SDUMLA\\IRIS(129_129)\\mixTrain"
validation_data_dir_nir = "C:\\Users\\Desktop\\SDUMLA\\IRIS(129_129)\\mixTest"
train_gen=custom_iterator(train_data_dir_nir, train_data_dir)
valid_gen=custom_iterator(validation_data_dir_nir, validation_data_dir)
nb_train_samples=2226
nb_validation_samples=1272
batch_size=10
Spe=nb_train_samples/ batch_size
valiStep=nb_validation_samples / batch_size

print ('compiling')
loss='categorical_crossentropy',
metrics=['accuracy'])

hist=modal1_1.fit_generator(train_gen,
steps_per_epoch = Spe,
epochs = 200,verbose = 1,
validation_data = valid_gen,
validation_steps = valiStep)


how can i deal with this problem? i have added dropouts to reduce overfitting. i don't know what mistake i am doing. can any one suggest the concept how i can merger two generators that are having different datasets, so it can serve my purpose.

closed as off-topic by Michael Chernick, kjetil b halvorsen, Peter Flom♦Jan 11 at 10:44

This question appears to be off-topic. The users who voted to close gave this specific reason:

• "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – Michael Chernick, kjetil b halvorsen, Peter Flom
If this question can be reworded to fit the rules in the help center, please edit the question.