Search Results
| Search type | Search syntax |
|---|---|
| Tags | [tag] |
| Exact | "words here" |
| Author |
user:1234 user:me (yours) |
| Score |
score:3 (3+) score:0 (none) |
| Answers |
answers:3 (3+) answers:0 (none) isaccepted:yes hasaccepted:no inquestion:1234 |
| Views | views:250 |
| Code | code:"if (foo != bar)" |
| Sections |
title:apples body:"apples oranges" |
| URL | url:"*.example.com" |
| Saves | in:saves |
| Status |
closed:yes duplicate:no migrated:no wiki:no |
| Types |
is:question is:answer |
| Exclude |
-[tag] -apples |
| For more details on advanced search visit our help page | |
Results tagged with word-embeddings
Search options not deleted
user 410763
Word embedding is the collective name for a set of language modeling and feature learning techniques in NLP where words are mapped to vectors of real numbers in a low dimensional space, relative to the vocabulary size.
3
votes
Accepted
Is concatenating a single integer sufficient for encoding positional embeddings in transform...
Good question.
One problem with your proposed method of encoding is that, for lengthy sequences, your positional encoding integer can end up being quite large, out-scaling the other dimensions.
This a …
- 156
5
votes
Intuitive explanation for summing the embedding and positional encoding in the Transformer's...
I've broken my head over this question for a long time.
Here's the best answer I came across, from redit.
TL;DR: It is intuitively possible that, in high dimensions, the word vectors form a smaller d …
- 156
6
votes
1
answer
149
views
What advantage do sinusoidal positional encodings have over binary positional encodings in t...
I've recently come across an article that discusses the reasons why large language models use sinusoidal functions to generate positional encodings — as per the famous paper Attention Is All You Need …
- 156