Questions tagged [embeddings]

The tag has no usage guidance.

22 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
1answer
218 views

What is the difference in the latent space of a variational autoencoder and a regular autoencoder?

Should VAEs be even used for non-generative tasks? If I were to use both models for embedding images, how would the embedding space differ on a structural level?
3
votes
0answers
246 views

Why researchers use conv1d for embeddings instead of dense layers?

In some papers (like Reinforcement learning for Vehicle Routing Problem), researchers use conv1d to embed the problem input into a hyperspace; for example, in solving TSP, they use conv1d on the (x,y) ...
2
votes
0answers
15 views

How to apply the diffusion maps when the matrix is PSD but not positivity preserving?

In order to apply the diffusion maps in a matrix $C\in\mathbb R^{n\times n}$ , that matrix must obey some restrictions, C is symmetric: $C_{ij} = C_{ji}$, C is positivity preserving (PP): $\forall ...
2
votes
0answers
72 views

Can you use VAEs to produce deep word embeddings?

There are many articles about applications of VAE such as image reconstruction, denoising, data compression / augmentation. However, I have not seen an example of embeddings for high dimensional data ...
2
votes
0answers
82 views

Recovering a distance matrix from nonnegative sparse correlation matrix?

After doing extensive literature research in all sorts of science I am completely puzzled. I am trying to find out what the state-of-the-art techniques would be to recover a (let's say euclidean) ...
2
votes
0answers
78 views

Is there an extension of PCA for data embedded in hyperbolic spaces?

I'm working on a project where we are embedding data into an n-dimensional Poincare ball similar to this paper. However, we'd like to take the additional step of reducing this data to a 2-dimensional ...
1
vote
0answers
8 views

Neural network embedding layers allowing multiple class-membership features

Is there a version of embedding layers for neural networks that allows for multiple class-membership features? Any frameworks that have implemented this? E.g. imagine we are trying to predict ...
1
vote
0answers
7 views

BERT for non-textual sequence data

I'm working on a deep learning solution for classifying sequence data that isn't raw text but rather entities (which have already been extracted from the text). I am currently using word2vec-style ...
1
vote
0answers
39 views

Are training-loss optimised embeddings useless? (help resolve a disagreement)

The aim We are training a feed forward neural network as a regressor, with the aim of using the activations of the final layer as a type of embedding vector to represent the input examples. The ...
1
vote
1answer
317 views

How do I add a missing word to a pretrained embedding?

I have a pretrained word embedding and want to add missing words to it. How exactly should I do that? I think to just randomly initialize the vector is not a good idea. I heard something about ...
1
vote
0answers
199 views

Machine learning on JSON/XML/DOM data

What's the best approach for machine learning on deeply hierarchical JSON/XML/DOM documents (not counting text nodes)? Say I want to recognize and generate documents similar to a training set of ...
1
vote
0answers
295 views

feature embedding for categorical features

I'm training a model and among the features, I have the language of the users. Right now I have done one-hot encoding on the language feature. But I think it would make more sense to have the language ...
1
vote
0answers
79 views

Derivation of local linear embedding

Might be a trivial question, but how do I solve for the following constrained optimization problem that appeared in local linear embedding? $$\min_{w_1,\cdots,w_k} \|x-\sum_{i=1}^k w_i x_i\|^2 \text{ ...
0
votes
0answers
4 views

Signal Embeddings using the skip-gram or CBOW model

So my work involves looking at a bunch of waveforms in the context of classifying events. I often am looking for new ways to represent my waveforms, and in my searching, I came across audio embeddings ...
0
votes
0answers
7 views

how to use contextual embeddings?

I've read about pretrained word embeddings, and I understand how to use them. Basically, if I have the word nail (for example), there is a look up table where I can use the embedding for that word. ...
0
votes
1answer
9 views

Reconstructing face from randomised embedding

It is fairly agreed in literature that from a given face-embedding (that is a vector of features values) it is possible, with a good amount of effort, to reconstruct the original face, (See here for ...
0
votes
0answers
36 views

node2vec: Intuition behind BFS resulting in embeddings that capture structural equivalence

In the node2vec paper1 it is mentioned that when using BFS to embed nodes, the results correspond to structural equivalence (i.e. nodes that are "bridge nodes" would get embedded close together) ...
0
votes
0answers
35 views

Understanding method of time series delay embedding

As discussed e.g. in the paper by Muskulus and Verduyn-Lunel http://www.math.leidenuniv.nl/reports/files/2009-12.pdf the delay vector reconstruction of time series for a dynamical model allows to ...
0
votes
1answer
71 views

Spectral embedding: interpretation of new dimensions

I'm trying to gain an intuition for the 2nd dimension in the spectral embedding of an S-shaped dataset as in this example: The 1st dimension seems to neatly capture the local similarity between ...
0
votes
0answers
17 views

Partitioning data for learning embeddings

I have high-dimensional events that I want to feed into an LSTM. I was planning to pre-learn a context embedding (like word2vec), as opposed to learning an embedding as the first layer in the network,...
0
votes
0answers
13 views

Embedding Dimension finds same projection direction with more projection dimensions

I have the following data setup: $$X \in \mathbb{R}^{n \times d}$$ $$Y \in \{1, ..., K\}^n$$ And I want to find a low-dimensional representation (something like PCA): $$A^r = embedding(X, Y, r)$$ ...
0
votes
2answers
2k views

Learning image embeddings using VGG and Word2Vec

Background: In word2vec we pass in a one-hot encoding of our target word into a simple neural network which is trained to predict context words from a window around our target. We eventually take the ...