Tag Info

0

You appear to be referring to the view of the computation graph provided by tensorboard or a similar visualization tool. Typically, these visualization tools don't draw every weight as a separate edge -- that would not really be feasible, since neural networks can have hundreds of millions to billions of parameters, which I doubt most plotting software ...

1

I thought it would be the same mechanism for pictures. In 1D convolution, each filter would be responsible for some particular pattern or semantic information. Take for example the following character aware encoder: Source: https://zhangruochi.com/Subword-Models/2019/12/19/ The final vector can be treated as the embedding for the word where each number ...

0

These results are the same for character n-grams as well. until a certain n value, accuracy will increase and then it starts to drop for both word n-grams and character n-grams. Reason for this is if the individual accuracy of a given n value is lower for a particular selected corpus size it has a slightly negative effect on combined feature selection ...

1

Sampling a language model is a good way of understanding what the model has learnt. You're right there's no point randomly sampling(with equal probability of each word at each time step t), then why have you even trained your model!? We rather sample based on the probability distribution of $\hat{y^{t}}$. Eg: say probability of $\hat{y_{i}^{t}}=0.16$ (...

2

There is no rule of thumb, it very much depends on how difficult your dataset is. f you have only one instance for a class, the model will certainly have the capacity to memorize that particular instance. If you have four of them, it will very likely happen as well. Including those in the training data probably would not do much harm, but you cannot expect ...

0

I hadn't realized this question was unanswered. If I were to attempt to answer my own question, we apply masks to the source data because after the data passes through the Encoder sublayer, there are values for the padding sequences. We don't need nor want the model to attend to these padding sequences, and so we mask them out. It's slightly different from ...

Top 50 recent answers are included