Questions tagged [embeddings]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
8 views

Using compositional data analysis to represent chemical compounds

I've recently got some insights about compositional data analysis, wondering whether it could be suitable for the framework I'm currently in. Recently, I've been very interested in trying to find some ...
1
vote
0answers
20 views

Definition of the word “embedding”

The mathematical definition of the word "embedding" requires the mapping to be injective, so in that context one speaks of, for example, embedding real numbers in complex numbers (ie, ...
0
votes
0answers
25 views

training embedding, autoencoder and classification. Not classification learning

I have implemented a network that has an autoencoder and after that a Multiple layer perceptron (MLP). Before autoencoder I used embedding layers. The embeddings’ outputs are inputs of autoencoder. I ...
3
votes
2answers
68 views

Embedding data into a larger dimension space

Embeddings or latent spaces are vector spaces that we embed our initial data into that for further processing. The benefit of doing so as far as I am aware, is to reduce the dimension. Often data has ...
0
votes
0answers
8 views

Input embedding layer keras (CBOW)

I have a question about the treatment that the embedding layer does to the inputs in keras. model.add(Embedding(input_dim, output_dim, input_length=max_length)) In an embedding layer, for a CBOW model,...
4
votes
0answers
155 views

If the curse of dimensionality exists, how does embedding search work?

The curse of dimensionality tells us if the dimension is high, the distance metric will stop working, i.e., everyone will be close to everyone. However, many machine learning retrieval systems rely on ...
0
votes
0answers
7 views

Kernel PCA embedding with the matrix inverse instead of square root

Given a set of points $X = \{x_1, x_2, \cdots, x_m\}$, one way of defining the Kernel PCA embedding for a new point $z$ is $K^{-1/2} [k(z, x_1), k(z, x_2), \cdots, k(z, x_m)]$ where $k$ is the kernel ...
0
votes
0answers
18 views

Word-embedding does not provide expected relations between words

I am trying to train a word embedding to a list of repeated sentences where only the subject changes. I expected that the generated vectors corresponding the subjects provide a strong correlation ...
0
votes
0answers
9 views

Systems outputting embedding vectors of multiple diverse top-N concepts?

In some classification tasks (e.g. object identification, text named entity recognition), sometimes we'd like to: have the system consider many potential output classes (>>10000 .. 100M), ...
0
votes
0answers
4 views

Graph to graph mapping

I am interested in learning a transformation of large graphs (> 1mio nodes, > 20mio edges) onto other graphs. I am aware of GNNs, but the embedding of the original graph into some $\mathbb{R}^n$ ...
0
votes
0answers
25 views

word embedding using Keras Embedding layer

I am learning using Keras Embedding layer to build embedding models. However, I failed to build a good embedding model. Can anyone help me check where I did wrong? Or not enough data to train? Data ...
0
votes
0answers
4 views

Why embedding layers can be regarded as a form of conditional computation?

Recently, I am reading the paper, "Outrageously Large Neural Network: The Sparsely-Gated Mixture-Of-Experts Layer". In the introduction section of the paper, authors mention about the ...
1
vote
1answer
28 views

Generating embeddings for languages without a written representation?

I'm considering the topic of generating an NLP Embedding for a language without a written standard or a significant corpus. I realized that as challenging as that is, it is still not as challenging as ...
1
vote
0answers
21 views

Is there an MDS/embedding algorithm that is more suitable to the goal of clustering a graph

I am testing ideas on clustering a particular graph. After testing a set of graph clustering/community detection algorithms I thought about mapping the graph to a vector space and using vector space ...
4
votes
1answer
90 views

ArcFace - How to compute $\cos(t+m)$ when $t+m > \pi$

I am trying to understand the ArcFace Implementation and I am stuck at one condition. If the $ \cos(t) > \cos(\pi -m)$ then $t + m > \pi$. In this case the way how we're computing $\cos(t+m)$ is ...
1
vote
0answers
23 views

Learn sentence embeddings from a sequence of token embeddings

I want to build a sentence classifier that takes the sentence as a sequence of token embeddings. I'm specifically interested in the methodology for learning the sentence embedding from the sequence of ...
1
vote
1answer
32 views

Extracting word embedding features of a sentence using Transformer-XL

As you know, there are several pre-trained models that we can use to extract word embeddings. As an example, I can use the following codes to retrieve word2vec features of my text: ...
0
votes
0answers
14 views

Extracting features from RNN networks

So we know that it is possible to extract features of the last layers from models such as AlexNet that are trained to classify images and use them for other vision tasks. That fact is true for ...
1
vote
0answers
50 views

How to pass an array of variable length to the input of the neural network?

I have a bunch of two-dimensional points that I want to feed into my neural network as an input. Those points are positions of the visible obstacles around my agent. The main challenge is that the ...
1
vote
0answers
30 views

Embeddings model: out of sample prediction with Keras for Collaborative Filtering

I have been trying to play with an example on Collaborative Filtering for Movie Recommendations (keras.io), which builds embedding layers for movies and users. Now, in a regular pre-trained word- and ...
1
vote
1answer
44 views

What's the best practice for dealing with OOV characters?

I have read on the advantages of using character-level language models over word-level ones. In particular, you don't have to deal with the problem of out of vocabulary (OOV) words, since characters ...
0
votes
1answer
18 views

What is the relationship between embeddings and deep metric learning?

Embeddings and deep metric learning seem architecturally identical. They both rely on using some hidden layer's vector representation of an input. What is the difference between the two? Are ...
4
votes
1answer
262 views

What is embedding? (in the context of dimensionality reduction)

In the context of dimensionality reduction one often uses word embedding, which seems to me a rather technical mathematical term, which rather stands out compared to the rest of the discussion, which ...
0
votes
0answers
96 views

Which dimensionality reduction technique works well for BERT sentence embeddings?

I'm trying to cluster hundreds of text documents so that each each cluster represents a distinct topic. Instead of using topic modeling (which I know I could do too), I want to follow a two-step ...
0
votes
0answers
48 views

dimension of input layer for embeddings in Keras

It is not clear to me whether there is any difference between specifying the input dimension Input(shape=(20,)) or not ...
0
votes
0answers
25 views

Is there an algorithm for placing 2-dimensional embeddings into a grid so they can be displayed?

I’m using PCA to reduce images down to 2d embeddings and I’d like to display the images in a grid. The Pudding did something like this with book covers, using tsne and a library called RasterFairy, by ...
1
vote
0answers
38 views

Why are we interested in gradient with respect to input?

I am learning about sampling methods for Deep Embedding Learning. I was reading an article named: "Sampling Matters in Deep Embedding Learning" (https://arxiv.org/abs/1706.07567). In the ...
2
votes
1answer
147 views

What is the difference between embedding in pure math and embedding in ML?

In ML the term "embedding" gets tossed around a lot and the term basically means the construction of a function that takes a high-dimensional vector to a low-dimensional vector in such a way ...
1
vote
2answers
32 views

Why are character level models considered less effective than word level models?

I have read that character level models need more computation power than word embeddings, and this is one of the major reasons for their less effectiveness, but i got curious because the word ...
0
votes
1answer
28 views

Why do Dense layers perform better than a mix of Conv Layers, Recurrent Layers on Sentiment Analysis with BERT emebddings?

I have used BERT to make embeddings out of the imdb review dataset and I am trying out some models to check their perfomance on sentiment analysis (0 for the bad reviews and 1 for the good ones). I ...
2
votes
1answer
64 views

Is there any paper about applications of Deep Metric Learning on regression problem?

I'm trying to solve a problem in the field of transfer learning, more specifically, domain adaption where both the source domain and target domain are labeled. Basically it's to predict the ...
1
vote
1answer
289 views

Neural network backpropagation to update inputs, not weights (e.g. fine-tuning embeddings)?

I recently re-read Stanford CS231N lecture notes on computer vision and backpropagation, and I came across this passage (emphasis mine): Note that (as is usually the case in Machine Learning) we ...
1
vote
0answers
24 views

Adam converges while SGD does not improve at all

I am trying to build a model based movie recommendation system with a neural network. The architecture looks as follows: ...
2
votes
1answer
549 views

Why BERT use learned positional embedding?

Compared with sinusoidal positional encoding used in Transformer, BERT's learned-lookup-table solution has 2 drawbacks in my mind: Fixed length Cannot reflect relative distance Could anyone please ...
3
votes
1answer
299 views

What are state of the art methods for creating embeddings for sets?

I want to create embeddings in $R^D$ for sets. So I want a function (probably a neural network) that takes in a set $ S = \{ s_1, \dots, s_n \} $ (and ideally of any size, so the number of elements ...
1
vote
3answers
51 views

General mathematical definition of a score

I understand what scores are in PCA, in particular this answer gives a good mathematical formulation: (Scores) are projections of the centred data in the linear space defined by the eigenvectors. ...
0
votes
1answer
91 views

Embedded markov chain example

I have an example of in my textbook of an "embeded markov chain", where I don't understand one step. Suppose that $(X_n)_{n\geq 0}$ is Markov$(\lambda, P)$. $\lambda$ is the initial distribution and ...
1
vote
1answer
69 views

Are the conditions of metric space satisfied in the latent space of a classification task?

Specifically, in the case of a neural network trained in a categorical classification task (cross-entropy loss function), does the final layer embedding space preserve the definition of distance ...
2
votes
0answers
34 views

What are embeddings in the context of machine learning?

I would like to find out an intuitive explanation of what are embeddings in the context of machine learning and neural networks. Is that essentially the same thing as a manifold? I've read a bunch of ...
0
votes
0answers
140 views

Using label encoder on a categorical feature that we want to embed

I have a dataset with feature that have very high cardinality, doing one-hot encoding is not an option because of memory limitations, so I am currently label encoding this feature and then I feed that ...
1
vote
0answers
43 views

t-sne embedding to medium-dimensions (e.g. 100 dimensions)?

I am using t-sne on 252 dimensional data to embed to lower-dimensions. I am curious to know if it is academically justifiable to embed it into medium dimensions such as 100 dimensions, or 80 ...
2
votes
0answers
17 views

Neural network embedding layers allowing multiple class-membership features

Is there a version of embedding layers for neural networks that allows for multiple class-membership features? Any frameworks that have implemented this? E.g. imagine we are trying to predict ...
2
votes
0answers
104 views

BERT for non-textual sequence data

I'm working on a deep learning solution for classifying sequence data that isn't raw text but rather entities (which have already been extracted from the text). I am currently using word2vec-style ...
1
vote
0answers
44 views

how to use contextual embeddings?

I've read about pretrained word embeddings, and I understand how to use them. Basically, if I have the word nail (for example), there is a look up table where I can use the embedding for that word. ...
0
votes
1answer
40 views

Reconstructing face from randomised embedding

It is fairly agreed in literature that from a given face-embedding (that is a vector of features values) it is possible, with a good amount of effort, to reconstruct the original face, (See here for ...
3
votes
1answer
1k views

Is the Keras Embedding layer dependent on the target label?

I learned how to 'use' the Keras Embedding layer, but I am not able to find any more specific information about the actual behavior and training process of this layer. For now, I understand that the ...
2
votes
0answers
24 views

How to apply the diffusion maps when the matrix is PSD but not positivity preserving?

In order to apply the diffusion maps in a matrix $C\in\mathbb R^{n\times n}$ , that matrix must obey some restrictions, C is symmetric: $C_{ij} = C_{ji}$, C is positivity preserving (PP): $\forall ...
0
votes
1answer
97 views

Spectral embedding: interpretation of new dimensions

I'm trying to gain an intuition for the 2nd dimension in the spectral embedding of an S-shaped dataset as in this example: The 1st dimension seems to neatly capture the local similarity between ...
8
votes
1answer
894 views

What is the intuition behind the positional cosine encoding in the transformer network?

I don't understand how adding the cosine encodings/functions to each of the dimension of the word vector embedding enables the network to "understand" where each word is situated in the sentence. ...
3
votes
0answers
123 views

Can you use VAEs to produce deep word embeddings?

There are many articles about applications of VAE such as image reconstruction, denoising, data compression / augmentation. However, I have not seen an example of embeddings for high dimensional data ...