New answers tagged natural-language
1
vote
Is pretraining on test set texts (without labels) ok?
TLDR: seems fine, which is weird. I'll check that the experiment code is correct.
This answer contains empirical experiments similar to my experiment with PCA here. We'll run a few experiments with ...
- 1,260
0
votes
using latent dirichlet allocation to reduce the number of dimensions in bag of words model?
EDIT:
So after some more research, it appears to me that what they (e.g. the SAM paper) refer to as "topic proportion" is actually alluding the normalized posterior Dirichlet parameters $\...
- 41
0
votes
Why are language modeling pre-training objectives considered unsupervised?
The original GPT paper seems to consider the language modeling objective semi-supervised:
Our work broadly falls under the category of semi-supervised
learning for natural language.
https://cdn....
- 101
0
votes
Why are the embeddings of tokens multiplied by $\sqrt D$ (note not divided by square root of D) in a transformer?
As mentioned in the transformer paper 'Attention is all you need', the embedding matrix in the input and the pre-softmax linear transformation in the linear layer outside the decoder block share the ...
- 1
0
votes
BERT masking scheme
In my opinion, why the chosen token is not always replaced by the special token [mask] is the following:
suppose the token x, and its context variable denoted c(x);
BERT model tries to learn the ...
- 1
-1
votes
Popular named entity resolution software
It's possible to have quite robust answers to record linkage / entity resolution problems using AI vectorisation and a vector database. An example which evaluates the DBLP and ACM data set benchmark ...
Top 50 recent answers are included