From this dataset with paragraphs, questions about these paragraphs and answers from these paragraphs, when there exists, I'm trying to predict the sentence where there is an answer. After unsupervised and supervised methods, which weren't really successful, I'm trying with neural networks which you can found in this GitHub repository. After splitting the sentences in a fixed size of ten I have created three features for each of the ten sentences : the cosine distance for each (question-sentence) algorithm, the euclidean one and root matching between the of the question and the roots of the sentences.

So I can have up to 30 inputs vectors in train_x, with the actual target sentence stored in train_y I tried to test two neural networks with 20 inputs: cosine-euclidean on the one hand and cosine-root on the other hand. With the following model:

# create model
model = Sequential()
model.add(Dense(12, input_dim=20, activation='relu'))
model.add(Dense(20, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
# Compile model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
# Fit the model
model.fit(train_x, train_y, epochs=150, batch_size=10)

But the accuracy rate is always quite slow and when I evaluate it with:

# evaluate the model
scores = model.evaluate(test_x, test_y)
print("\n%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))

The first one had an accuracy of 25,37% and the second one of acc: 26.33%

So I'm wondering who is the culprit: is the distance who even if it seems to be good at finding the answer when there is one in one sentence seems not to be a really good feature when there is none? Or is it the root matching feature or is it my neural network architecture ?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.