New answers tagged

1

You can do one of three things: try different approaches still, like RNNs, CNNs, word vectorization... depending on your sample size and on your problem, they can be quite beneficial make an ensemble of the models you used. Simple model averaging increases generalizability and can easily improve performance in the training and validation sets as well. On ...


0

It is not really important in the encoder, but it plays a crucial role in the decoder. At the training time, you use the target sentence in the following way (with a 5-token sentence $w_1, \ldots, w_5$): [BOS] w₁ w₂ w₃ w₄ w₆ ↓ ↓ ↓ ↓ ↓ ↓ ┌─────────────────────────────┐ │ DECODER │ └─────────────────────────────┘ ...


1

One of the reasons for popularity of deep neural networks, is that they are able to do automatic feature extraction from the data. Traditionally, when solving some data problem, you would need to gather the data, clean it, and extract some features that would help your model to learn something from the data. Those features could be different transformations ...


4

It isn't really clear, but I think what he is doing is weighing words found under "predefined" topic tags in a discussion board, and then weighing those words (X1000) in the sampling process of LDA. For example, if I search stats.stackexchange under the tag "natural-language" and create a vocabulary of, word : # times word appeared, and ...


0

The number of features does not matter as long as the predictive power of the features is high enough. For example, if the data set is a collection of news articles, one feature contains tags and the response variable is the category, one can expect a reasonable model quality (assuming descriptive tags, of course). On the other hand, if the predictive power ...


2

There is no need to output embeddings nor one-hot vectors. After all, a tokenizer should work with any words, not only those you have in a vocabulary for the one-hot encoding. Tokenization is usually formualate as a sequence-labeling problem. The input is a character sequence and you label you label each character by assigning one of following classes: ...


1

In general, the data at inference time should be pre-proprocessed the same way at the training time. Although the LSTMs in theory generalize for arbitrarily long output, in practice, they usually generalize only slightly beyond the scope of what was seen in the training data. With summarization, the reason for truncating the input is, first of all, saving ...


Top 50 recent answers are included