# Tag Info

1

why does masking requires us to sample (take only 15%) of the words? I think this guarantees that only about 1/7 of the words are masked, which is just like the window size in word2vec. That is on average in BERT we use 7 words as context to predict one word. The more words we mask the smaller the "window size" and the smaller the context. Google ...

0

Ane model would be to estimate the probability of getting a certain diagnosis (per time frame) for each of the categories of interest. Then just draw samples from these distribution to generate diagnosises. A simple version could take just agegroup, gender as basis for categories. More advanced versions could include past medical history.

1

A high cross entropy and a high accuracy are not necessarily inconsistent results. Here's a simple example. Suppose you have a problem with 10 classes, and a data set with 1 sample of each class. In 9 of the samples, the correct class has prediction $0.85$, but in one class it has prediction $0.001$. The cross-entropy loss in this case is almost $0.84$. This ...

0

In keras.layers.LSTM(units, activation='tanh', ....), the units refers to the dimensionality or length of the hidden state or the length of the activation vector passed on the next LSTM cell/unit - the next LSTM cell/unit is the "green picture above with the gates etc from http://colah.github.io/posts/2015-08-Understanding-LSTMs/ The next LSTM cell/unit ...

1

$x$ is the input to the layer, it's not encoded as anything if you don't encode it. $W_{xh}$ is a matrix. You can interpret it as a rotation matrix that takes $x$ to the space of $h_t$.

0

In addition to the tools mentioned by @BCjuan, there is also. https://github.com/majianjia/nnom a platform independent inference engine for neural networks. It takes a Keras model as input. X-CUBE-AI from ST Microelectronics. It is however proprietary and only supports their devices.

Top 50 recent answers are included